Abstract:Learning the design patterns of proteins from sequences across evolution may have promise toward generative protein design. However it is unknown whether language models, trained on sequences of natural proteins, will be capable of more than memorization of existing protein families. Here we show that language models generalize beyond natural proteins to generate de novo proteins. We focus on two protein design tasks: fixed backbone design where the structure is specified, and unconstrained generation where the structure is sampled from the model. Remarkably although the models are trained only on sequences, we find that they are capable of designing structure. A total of 228 generated proteins are evaluated experimentally with high overall success rates (152/228 or 67%) in producing a soluble and monomeric species by size exclusion chromatography. Out of 152 experimentally successful designs, 35 have no significant sequence match to known natural proteins. Of the remaining 117, sequence identity to the nearest sequence match is at median 27%, below 20% for 6 designs, and as low as 18% for 3 designs. For fixed backbone design, the language model generates successful designs for each of eight experimentally evaluated artificially created fixed backbone targets. For unconstrained generation, sampled proteins cover diverse topologies and secondary structure compositions, and have high experimental success rate (71/129 or 55%). The designs reflect deep patterns linking sequence and structure, including motifs that occur in related natural structures, and motifs that are not observed in similar structural contexts in known protein families. The results show that language models, though only trained on sequences, learn a deep grammar that enables the design of protein structure, extending beyond natural proteins.

Tpgen: a language model for stable protein design with a specific topology structure

Global-Context Aware Generative Protein Design

Generative De Novo Protein Design with Global Context

PB-GPT: An innovative GPT-based model for protein backbone generation

ProtGPT2 is a deep unsupervised language model for protein design

Protein Design with StructureGPT: a Deep Learning Model for Protein Structure-to-Sequence Translation

A high-level programming language for generative protein design

InstructPLM: Aligning Protein Language Models to Follow Protein Structure Instructions

Language models generalize beyond natural proteins

Designing proteins with language models

OPUS-Design: Designing Protein Sequence from Backbone Structure with 3DCNN and Protein Language Model

Proximal Exploration for Model-guided Protein Sequence Design

Efficient Generation of Protein Pockets with PocketGen

Pro-PRIME: A general Temperature-Guided Language model to engineer enhanced Stability and Activity in Proteins

Functional Geometry Guided Protein Sequence and Backbone Structure Co-Design

Generating Novel, Designable, and Diverse Protein Structures by Equivariantly Diffusing Oriented Residue Clouds

Structure-informed Language Models Are Protein Designers

ProteinGPT: Multimodal LLM for Protein Property Prediction and Structure Understanding

A general temperature-guided language model to design proteins of enhanced stability and activity

A probabilistic view of protein stability, conformational specificity, and design

Out of Many, One: Designing and Scaffolding Proteins at the Scale of the Structural Universe with Genie 2