Abstract:One of the challenges in a viral pandemic is the emergence of novel variants with different phenotypical characteristics. An ability to forecast future viral individuals at the sequence level enables advance preparation by characterizing the sequences and closing vulnerabilities in current preventative and therapeutic methods. In this article, we explore, in the context of a viral pandemic, the problem of generating complete instances of undiscovered viral protein sequences, which have a high likelihood of being discovered in the future using protein language models. Current approaches to training these models fit model parameters to a known sequence set, which does not suit pandemic forecasting as future sequences differ from known sequences in some respects. To address this, we develop a novel method, called PandoGen, to train protein language models towards the pandemic protein forecasting task. PandoGen combines techniques such as synthetic data generation, conditional sequence generation, and reward-based learning, enabling the model to forecast future sequences, with a high propensity to spread. Applying our method to modeling the SARS-CoV-2 Spike protein sequence, we find empirically that our model forecasts twice as many novel sequences with five times the case counts compared to a model that is 30× larger. Our method forecasts unseen lineages months in advance, whereas models 4× and 30× larger forecast almost no new lineages. When trained on data available up to a month before the onset of important Variants of Concern, our method consistently forecasts sequences belonging to those variants within tight sequence budgets. Viral protein sequences play a pivotal role in the spread of a pandemic. As the virus evolves, so do the viral proteins, increasing the potency of the virus. Knowledge of future viral protein sequences can be invaluable because it allows us to test the efficacy of preventative and treatment methods against future changes to the virus, and tailor them to such changes early. We attempt to forecast viral proteins ahead of time. Making such predictions is very challenging and complex because the prediction target is a sequence with thousands of positions, and a single mis-predicted sequence position may invalidate the entire prediction. Also, as the virus continues to evolve, the data available to train models becomes obsolete. Addressing these challenges, we create a novel approach to train models of the SARS-CoV-2 Spike protein, that are especially tailored to forecasting future sequences. Models trained using this approach outperform existing approaches in their effectiveness. In addition, our method can train models to forecast important pandemic variants ahead of time.

Running Ahead of Evolution - AI Based Simulation for Predicting Future High-Risk SARS-CoV-2 Variants

Variation and Evolution Analysis of SARS-CoV-2 Using Self-Game Sequence Optimization

TEMPO: A Transformer-Based Mutation Prediction Framework for SARS-CoV-2 Evolution

Evolution-guided Large Language Model is a Predictor of Virus Mutation Trends

Predicting Natural Evolution in the RBD Region of the Spike Glycoprotein of SARS-CoV-2 by Machine Learning

Predicting the immune escape of SARS-CoV-2 neutralizing antibodies upon mutation

A Computer Simulation of SARS-CoV-2 Mutation Spectra for Empirical Data Characterization and Analysis

Mapping the Evolutionary Space of SARS-CoV-2 Variants to Anticipate Emergence of Subvariants Resistant to COVID-19 Therapeutics

Modeling the trajectory of SARS-CoV-2 spike protein evolution in continuous latent space using a neural network and Gaussian process

Early computational detection of potential high-risk SARS-CoV-2 variants

Predicting the antigenic evolution of SARS-COV-2 with deep learning

Computational prediction of the effect of amino acid changes on the binding affinity between SARS-CoV-2 spike RBD and human ACE2

Machine Learning Driven Simulations of SARS-CoV-2 Fitness Landscape

Topological AI forecasting of future dominating viral variants

An Interpretable Deep Learning Model for Predicting the Risk of Severe COVID-19 from Spike Protein Sequence

Modeling Viral Evolution: A Novel SIRSVIDE Framework with Application to SARS-CoV-2 Dynamics

Deep learning for discriminating non-trivial conformational changes in molecular dynamics simulations of SARS-CoV-2 spike-ACE2

VarEPS: an evaluation and prewarning system of known and virtual variations of SARS-CoV-2 genomes

PandoGen: Generating complete instances of future SARS-CoV-2 sequences using Deep Learning

SARITA: A Large Language Model for Generating the S1 Subunit of the SARS-CoV-2 Spike Protein

A prediction of mutations in infectious viruses using artificial intelligence