Abstract:We describe attempts to design protein sequences by inverting the protein structure prediction algorithm ESMFold. State-of-the-art protein structure prediction methods achieve high accuracy by relying on evolutionary patterns derived from either multiple sequence alignments (AlphaFold, RosettaFold) or pretrained protein language models (PLMs; ESMFold, OmegaFold). In principle, by inverting these networks, protein sequences can be designed to fulfill one or more design objectives, such as high prediction confidence, predicted protein binding, or other geometric constraints that can be expressed with loss functions. In practice, sequences designed using an inverted AlphaFold model, termed AFDesign, contain unnatural sequence profiles shown to express poorly, whereas an inverted RosettaFold network has been shown to be sensitive to adversarial sequences. Here, we demonstrate that these limitations do not extend to neural networks that include PLMs, such as ESMFold. Using an inverted ESMFold model, termed ESM-Design, we generated sequences with profiles that are both more native-like and more likely to express than sequences generated using AFDesign, but less likely to express than sequences rescued by the structure-based design method ProteinMPNN. However, the safeguard offered by the PLM came with steep increases in memory consumption, preventing proteins greater than 150 residues from being modeled on a single GPU with 80GB VRAM. During this investigation, we also observed the role played by different sequence initialization schemes, with random sampling of discrete amino acids improving convergence and model quality over any continuous random initialization method. Finally, we showed how this approach can be used to introduce sequence and structure diversification in small proteins such as ubiquitin, while respecting the sequence conservation of active site residues. Our results highlight the effects of architectural differences between structure prediction networks on zero-shot protein design.

All-Atom Protein Generation with Latent Diffusion

Global-Context Aware Generative Protein Design

Generative De Novo Protein Design with Global Context

A Latent Diffusion Model for Protein Structure Generation

PRO-LDM: Protein Sequence Generation with a Conditional Latent Diffusion Model

P(all-atom) Is Unlocking New Path For Protein Design

Generating Novel, Designable, and Diverse Protein Structures by Equivariantly Diffusing Oriented Residue Clouds

Protein structure generation via folding diffusion

Illuminating protein space with a programmable generative model

Protein generation with evolutionary diffusion: sequence is all you need

DPLM-2: A Multimodal Diffusion Protein Language Model

Diffusion Language Models Are Versatile Protein Learners

Secondary Structure-Guided Novel Protein Sequence Generation with Latent Graph Diffusion

Protein A-like Peptide Design Based on Diffusion and ESM2 Models

Full-Atom Peptide Design with Geometric Latent Diffusion

Structure Language Models for Protein Conformation Generation

Sampling Conformational Ensembles of Highly Dynamic Proteins via Generative Deep Learning

A conditional protein diffusion model generates artificial programmable endonuclease sequences with enhanced activity

Atom-by-atom protein generation and beyond with language models

Sequence-Augmented SE(3)-Flow Matching For Conditional Protein Backbone Generation

ESMFold Hallucinates Native-Like Protein Sequences