Abstract:Bioinformatics has witnessed a paradigm shift with the increasing integration of artificial intelligence (AI), particularly through the adoption of foundation models (FMs). These AI techniques have rapidly advanced, addressing historical challenges in bioinformatics such as the scarcity of annotated data and the presence of data noise. FMs are particularly adept at handling large-scale, unlabeled data, a common scenario in biological contexts due to the time-consuming and costly nature of experimentally determining labeled data. This characteristic has allowed FMs to excel and achieve notable results in various downstream validation tasks, demonstrating their ability to represent diverse biological entities effectively. Undoubtedly, FMs have ushered in a new era in computational biology, especially in the realm of deep learning. The primary goal of this survey is to conduct a systematic investigation and summary of FMs in bioinformatics, tracing their evolution, current research status, and the methodologies employed. Central to our focus is the application of FMs to specific biological problems, aiming to guide the research community in choosing appropriate FMs for their research needs. We delve into the specifics of the problem at hand including sequence analysis, structure prediction, function annotation, and multimodal integration, comparing the structures and advancements against traditional methods. Furthermore, the review analyses challenges and limitations faced by FMs in biology, such as data noise, model explainability, and potential biases. Finally, we outline potential development paths and strategies for FMs in future biological research, setting the stage for continued innovation and application in this rapidly evolving field. This comprehensive review serves not only as an academic resource but also as a roadmap for future explorations and applications of FMs in biology.

Foundation models in molecular biology

LucaOne: Generalized Biological Foundation Model with Unified Nucleic Acid and Protein Language

Foundation models for bioinformatics

Large-scale foundation model on single-cell transcriptomics

Progress and Opportunities of Foundation Models in Bioinformatics

The Development of AI Foundation Models for Single-Cell Transcriptomics

The Future of Molecular Studies Through the Lens of Large Language Models.

Large-scale Foundation Models and Generative AI for BigData Neuroscience

Interpretable RNA Foundation Model from Unannotated Data for Highly Accurate RNA Structure and Function Predictions

Deciphering RNA regulation with a foundation language model

How do Large Language Models understand Genes and Cells

Benchmarking DNA Foundation Models for Genomic Sequence Classification

scLong: A Billion-Parameter Foundation Model for Capturing Long-Range Gene Context in Single-Cell Transcriptomics

Integration of protein and coding sequences enables mutual augmentation of the language model

DGRNA: a long-context RNA foundation model with bidirectional attention Mamba2

Orthrus: Towards Evolutionary and Functional RNA Foundation Models

Bridging biomolecular modalities for knowledge transfer in bio-language models

Accurate and General DNA Representations Emerge from Genome Foundation Models at Scale