Abstract:Drug discovery typically consists of multiple steps, including identifying a target protein key to a disease's etiology, validating that interacting with this target could prevent symptoms or cure the disease, discovering a small molecule or biologic therapeutic to interact with it, and optimizing the candidate molecule through a complex landscape of required properties. Drug discovery related tasks often involve prediction and generation while considering multiple entities that potentially interact, which poses a challenge for typical AI models. For this purpose we present MAMMAL - Molecular Aligned Multi-Modal Architecture and Language - a method that we applied to create a versatile multi-task multi-align foundation model that learns from large-scale biological datasets (2 billion samples) across diverse modalities, including proteins, small molecules, and genes. We introduce a prompt syntax that supports a wide range of classification, regression, and generation tasks. It allows combining different modalities and entity types as inputs and/or outputs. Our model handles combinations of tokens and scalars and enables the generation of small molecules and proteins, property prediction, and transcriptomic lab test predictions. We evaluated the model on 11 diverse downstream tasks spanning different steps within a typical drug discovery pipeline, where it reaches new SOTA in 9 tasks and is comparable to SOTA in 2 tasks. This performance is achieved while using a unified architecture serving all tasks, in contrast to the original SOTA performance achieved using tailored architectures. The model code and pretrained weights are publicly available at <a class="link-external link-https" href="https://github.com/BiomedSciAI/biomed-multi-alignment" rel="external noopener nofollow">this https URL</a> and <a class="link-external link-https" href="https://huggingface.co/ibm/biomed.omics.bl.sm.ma-ted-458m" rel="external noopener nofollow">this https URL</a>.

ProtMamba: a homology-aware but alignment-free protein state space model

Multiple Conformational States Assembly of Multidomain Proteins Using Evolutionary Algorithm Based on Structural Analogues and Sequential Homologues

learnMSA2: deep protein multiple alignments with large language and hidden Markov models

PTM-Mamba: A PTM-Aware Protein Language Model with Bidirectional Gated Mamba Blocks

Design Proteins Using Large Language Models: Enhancements and Comparative Analyses

Protein-Mamba: Biological Mamba Models for Protein Function Prediction

Generative power of a protein language model trained on multiple sequence alignments

Enhancing predictions of protein stability changes induced by single mutations using MSA-based language models

Structure-conditioned masked language models for protein sequence design generalize beyond the native sequence space

Pairing interacting protein sequences using masked language modeling

So ManyFolds, So Little Time: Efficient Protein Structure Prediction With pLMs and MSAs

InstructPLM: Aligning Protein Language Models to Follow Protein Structure Instructions

Embedding-based alignment: combining protein language models with dynamic programming alignment to detect structural similarities in the twilight-zone

Designing diverse and high-performance proteins with a large language model in the loop

MAMMAL -- Molecular Aligned Multi-Modal Architecture and Language

fastMSA: Accelerating Multiple Sequence Alignment with Dense Retrieval on Protein Language

Robust deep learning based protein sequence design using ProteinMPNN

A-Prot: protein structure modeling using MSA transformer

Efficient Inference, Training, and Fine-tuning of Protein Language Models

Protein language models trained on multiple sequence alignments learn phylogenetic relationships

ProGen2: Exploring the Boundaries of Protein Language Models