Abstract:One of the challenges in a viral pandemic is the emergence of novel variants with different phenotypical characteristics. An ability to forecast future viral individuals at the sequence level enables advance preparation by characterizing the sequences and closing vulnerabilities in current preventative and therapeutic methods. In this article, we explore, in the context of a viral pandemic, the problem of generating complete instances of undiscovered viral protein sequences, which have a high likelihood of being discovered in the future using protein language models. Current approaches to training these models fit model parameters to a known sequence set, which does not suit pandemic forecasting as future sequences differ from known sequences in some respects. To address this, we develop a novel method, called PandoGen, to train protein language models towards the pandemic protein forecasting task. PandoGen combines techniques such as synthetic data generation, conditional sequence generation, and reward-based learning, enabling the model to forecast future sequences, with a high propensity to spread. Applying our method to modeling the SARS-CoV-2 Spike protein sequence, we find empirically that our model forecasts twice as many novel sequences with five times the case counts compared to a model that is 30× larger. Our method forecasts unseen lineages months in advance, whereas models 4× and 30× larger forecast almost no new lineages. When trained on data available up to a month before the onset of important Variants of Concern, our method consistently forecasts sequences belonging to those variants within tight sequence budgets. Viral protein sequences play a pivotal role in the spread of a pandemic. As the virus evolves, so do the viral proteins, increasing the potency of the virus. Knowledge of future viral protein sequences can be invaluable because it allows us to test the efficacy of preventative and treatment methods against future changes to the virus, and tailor them to such changes early. We attempt to forecast viral proteins ahead of time. Making such predictions is very challenging and complex because the prediction target is a sequence with thousands of positions, and a single mis-predicted sequence position may invalidate the entire prediction. Also, as the virus continues to evolve, the data available to train models becomes obsolete. Addressing these challenges, we create a novel approach to train models of the SARS-CoV-2 Spike protein, that are especially tailored to forecasting future sequences. Models trained using this approach outperform existing approaches in their effectiveness. In addition, our method can train models to forecast important pandemic variants ahead of time.

Variational autoencoder for design of synthetic viral vector serotypes

Generative AAV capsid diversification by latent interpolation

Ig-VAE: Generative modeling of protein structure by direct 3D coordinate generation

Leveraging Synthetic Virology for the Rapid Engineering of Vesicular Stomatitis Virus (VSV)

Vaxformer: Antigenicity-controlled Transformer for Vaccine Design Against SARS-CoV-2

Protein design for evaluating vaccines against future viral variation

ProtWave-VAE: Integrating Autoregressive Sampling with Latent-Based Inference for Data-Driven Protein Design

Towards AI-designed genomes using a variational autoencoder

Variational auto-encoding of protein sequences

In Silico Design of a Trans-Amplifying RNA-Based Vaccine against SARS-CoV-2 Structural Proteins

Structure-guided evolution of antigenically distinct adeno-associated virus variants for immune evasion

Cell‐engineered virus‐mimetic nanovesicles for vaccination against enveloped viruses

Deep diversification of an AAV capsid protein by machine learning

Comprehensive AAV capsid fitness landscape reveals a viral gene and enables machine-guided design

Antiviral Peptide-Generative Pre-trained Transformer (AVP-GPT): A Deep Learning-Powered Model for Antiviral Peptide Design with High-Throughput Discovery and Exceptional Potency

Computer-Aided Directed Evolution Generates Novel AAV Variants with High Transduction Efficiency

How to Hallucinate Functional Proteins

Modeling the trajectory of SARS-CoV-2 spike protein evolution in continuous latent space using a neural network and Gaussian process

Development of novel SARS-CoV-2 viral vectors

PandoGen: Generating complete instances of future SARS-CoV-2 sequences using Deep Learning