Abstract:Language models are thriving, powering conversational agents that assist and empower humans to solve a number of tasks. Recently, these models were extended to support additional modalities including vision, audio and video, demonstrating impressive capabilities across multiple domains including healthcare. Still, conversational agents remain limited in biology as they cannot yet fully comprehend biological sequences. On the other hand, high-performance foundation models for biological sequences have been built through self-supervision over sequencing data, but these need to be fine-tuned for each specific application, preventing transfer and generalization between tasks. In addition, these models are not conversational which limits their utility to users with coding capabilities. In this paper, we propose to bridge the gap between biology foundation models and conversational agents by introducing ChatNT, the first multimodal conversational agent with an advanced understanding of biological sequences. ChatNT achieves new state-of-the-art results on the Nucleotide Transformer benchmark while being able to solve all tasks at once, in English, and to generalize to unseen questions. In addition, we have curated a new set of more biologically relevant instructions tasks from DNA, RNA and proteins, spanning multiple species, tissues and biological processes. ChatNT reaches performance on par with state-of-the-art specialized methods on those tasks. We also present a novel perplexity-based technique to help calibrate the confidence of our model predictions. Our framework for genomics instruction-tuning can be easily extended to more tasks and biological data modalities (e.g. structure, imaging), making it a widely applicable tool for biology. ChatNT is the first model of its kind and constitutes an initial step towards building generally capable agents that understand biology from first principles while being accessible to users with no coding background.

Multimodal learning of transcriptomes and text enables interactive single-cell RNA-seq data exploration with natural-language chats

ChatCell: Facilitating Single-Cell Analysis with Natural Language

scChat: A Large Language Model-Powered Co-Pilot for Contextualized Single-Cell RNA Sequencing Analysis

A sandbox for prediction and integration of DNA, RNA, and proteins in single cells

CELLama: Foundation Model for Single Cell and Spatial Transcriptomics by Cell Embedding Leveraging Language Model Abilities

scReader: Prompting Large Language Models to Interpret scRNA-seq Data

CellChat for systematic analysis of cell-cell communication from single-cell transcriptomics

Inference and analysis of cell-cell communication using CellChat

Integrating T-cell Receptor and Transcriptome for Large-Scale Single-Cell Immune Profiling Analysis

scKEPLM: Knowledge enhanced large-scale pre-trained language model for single-cell transcriptomics

Cellenium-a scalable and interactive visual analytics app for exploring multimodal single-cell data

ChatNT: A Multimodal Conversational Agent for DNA, RNA and Protein Tasks

Harnessing Agent-Based Modeling in CellAgentChat to Unravel Cell-Cell Interactions from Single-Cell Data

CrossMP: Enabling Cross-Modality Translation between Single-Cell RNA-Seq and Single-Cell ATAC-Seq through Web-Based Portal

Cell2Sentence: Teaching Large Language Models the Language of Biology

Scmulan: a Multitask Generative Pre-Trained Language Model for Single-Cell Analysis

Comparative analysis of cell-cell communication at single-cell resolution

Integrated analysis of multimodal single-cell data

Contrastive Learning for Robust Cell Annotation and Representation from Single-Cell Transcriptomics

Generalizable Morphological Profiling of Cells by Interpretable Unsupervised Learning

ChatMol: Interactive Molecular Discovery with Natural Language