Abstract:Recent studies have demonstrated the strong empirical performance of diffusion models on discrete sequences across domains from natural language to biological sequence generation. For example, in the protein inverse folding task, conditional diffusion models have achieved impressive results in generating natural-like sequences that fold back into the original structure. However, practical design tasks often require not only modeling a conditional distribution but also optimizing specific task objectives. For instance, we may prefer protein sequences with high stability. To address this, we consider the scenario where we have pre-trained discrete diffusion models that can generate natural-like sequences, as well as reward models that map sequences to task objectives. We then formulate the reward maximization problem within discrete diffusion models, analogous to reinforcement learning (RL), while minimizing the KL divergence against pretrained diffusion models to preserve naturalness. To solve this RL problem, we propose a novel algorithm, DRAKES, that enables direct backpropagation of rewards through entire trajectories generated by diffusion models, by making the originally non-differentiable trajectories differentiable using the Gumbel-Softmax trick. Our theoretical analysis indicates that our approach can generate sequences that are both natural-like and yield high rewards. While similar tasks have been recently explored in diffusion models for continuous domains, our work addresses unique algorithmic and theoretical challenges specific to discrete diffusion models, which arise from their foundation in continuous-time Markov chains rather than Brownian motion. Finally, we demonstrate the effectiveness of DRAKES in generating DNA and protein sequences that optimize enhancer activity and protein stability, respectively, important tasks for gene therapies and protein-based therapeutics.

DiscDiff: Latent Diffusion Model for DNA Sequence Generation

Absorb & Escape: Overcoming Single Model Limitations in Generating Genomic Sequences

DiffSeqMol: A Non-Autoregressive Diffusion-Based Approach for Molecular Sequence Generation and Optimization

Designing DNA With Tunable Regulatory Activity Using Discrete Diffusion

Latent Diffusion Models for Controllable RNA Sequence Generation

TaxDiff: Taxonomic-Guided Diffusion Model for Protein Sequence Generation

PRO-LDM: Protein Sequence Generation with a Conditional Latent Diffusion Model

Protein generation with evolutionary diffusion: sequence is all you need

Secondary Structure-Guided Novel Protein Sequence Generation with Latent Graph Diffusion

Bridging the Gap between Learning and Inference for Diffusion-Based Molecule Generation

DNA-Diffusion: Leveraging Generative Models for Controlling Chromatin Accessibility and Gene Expression via Synthetic Regulatory Elements

A versatile informative diffusion model for single-cell ATAC-seq data generation and analysis

Dirichlet Diffusion Score Model for Biological Sequence Generation

Fine-Tuning Discrete Diffusion Models via Reward Optimization with Applications to DNA and Protein Design

scDiffusion: conditional generation of high-quality single-cell data using diffusion model

Dirichlet Flow Matching with Applications to DNA Sequence Design

Discrete Copula Diffusion

Scalable DNA Feature Generation and Transcription Factor Binding Prediction via Deep Surrogate Models

Utilizing Latent Diffusion Model to Accelerate Sampling Speed and Enhance Text Generation Quality

Towards Joint Sequence-Structure Generation of Nucleic Acid and Protein Complexes with SE(3)-Discrete Diffusion

LDMol: Text-to-Molecule Diffusion Model with Structurally Informative Latent Space