Abstract:Protein sequence design has seen significant advances through discrete diffusion and autoregressive approaches, yet the potential of continuous diffusion remains underexplored. Here, we present DiMA, a latent diffusion framework that operates on protein language model representations. Through systematic exploration of architectural choices and diffusion components, we develop a robust methodology that generalizes across multiple protein encoders ranging from 8M to 3B parameters. We demonstrate that our framework achieves consistently high performance across sequence-only (ESM-2, ESMc), dual-decodable (CHEAP), and multimodal (SaProt) representations using the same architecture and training approach. We extensively evaluate existing methods alongside DiMA using multiple metrics across two protein modalities, covering quality, diversity, novelty, and distribution matching of generated proteins. DiMA consistently produces novel, high-quality and diverse protein sequences and achieves strong results compared to baselines such as autoregressive, discrete diffusion and flow matching language models. The model demonstrates versatile functionality, supporting conditional generation tasks including protein family-generation, motif scaffolding and infilling, and fold-specific sequence design. This work provides a universal continuous diffusion framework for protein sequence generation, offering both architectural insights and practical applicability across various protein design scenarios.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the application and optimization of continuous diffusion models in the field of protein sequence generation. Specifically, the authors developed a latent diffusion model named DiMA, aiming to generate high - quality and diverse protein sequences by manipulating the representations of protein language models (pLMs). ### Main problems: 1. **Limitations of existing methods**: - The field of protein sequence design has made significant progress through discrete diffusion models and autoregressive models. - However, the potential of continuous diffusion models in protein sequence generation has not been fully explored. - Existing attempts have mainly focused on specific protein representations or conditional tasks, failing to fully exploit the potential of continuous diffusion models. 2. **Generality and adaptability of the model**: - A framework that can work effectively on different protein encoders (from 8M to 3B parameters) is required. - The model needs to be able to handle multiple types of protein representations, including sequence - only, dual - decodable, and multimodal representations. ### Solutions: - **Develop DiMA**: A continuous latent diffusion model based on pLM representations. - **Systematically explore architecture choices and diffusion components**: Ensure the generality and efficiency of the model on different encoders. - **Evaluate and compare**: Evaluate the performance of existing methods and DiMA through multiple metrics (quality, diversity, novelty, and distribution matching) to verify its superiority. ### Specific contributions: 1. **Wide applicability**: DiMA can work effectively on different pLM representations (such as ESM - 2, CHEAP, SaProt). 2. **Robust architecture**: Through systematic architecture selection and diffusion component optimization, a general method applicable to encoders with 8M to 3B parameters has been established. 3. **High performance**: Despite having only 35M parameters, DiMA performs excellently in multiple benchmark tests and even outperforms specialized models. 4. **High - quality generation**: Experiments have proven that DiMA can generate protein sequences with high structural quality, diversity, and novelty. 5. **Flexibility**: A single architecture can be adapted to conditional tasks and backbone generation through lightweight modifications, demonstrating a unified path for protein sequence generation. Through these efforts, DiMA provides a general continuous diffusion framework for protein sequence generation, which not only has important insights in architecture but also has wide - ranging value in practical applications.

Diffusion on language model encodings for protein sequence generation

AMP-Diffusion: Integrating Latent Diffusion with Protein Language Models for Antimicrobial Peptide Generation

A Unified Conditional Diffusion Framework for Dual Protein Targets Based Bioactive Molecule Generation

PRO-LDM: Protein Sequence Generation with a Conditional Latent Diffusion Model

Diffusion Language Models Are Versatile Protein Learners

Protein generation with evolutionary diffusion: sequence is all you need

DPLM-2: A Multimodal Diffusion Protein Language Model

MeMDLM: De Novo Membrane Protein Design with Masked Discrete Diffusion Protein Language Models

A Latent Diffusion Model for Protein Structure Generation

DiffSDS: A language diffusion model for protein backbone inpainting under geometric conditions and constraints

Improving diffusion-based protein backbone generation with global-geometry-aware latent encoding

DisCo-Diff: Enhancing Continuous Diffusion Models with Discrete Latents

Fine-Tuning Discrete Diffusion Models via Reward Optimization with Applications to DNA and Protein Design

Protein Design with Guided Discrete Diffusion

Latent Diffusion for Language Generation

ExEnDiff: An Experiment-guided Diffusion model for protein conformational Ensemble generation

Graph Denoising Diffusion for Inverse Protein Folding

LaGDif: Latent Graph Diffusion Model for Efficient Protein Inverse Folding with Self-Ensemble

Unified Generation, Reconstruction, and Representation: Generalized Diffusion with Adaptive Latent Encoding-Decoding

Protein A-like Peptide Design Based on Diffusion and ESM2 Models

Scaling Diffusion Language Models via Adaptation from Autoregressive Models