Abstract:Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has caused an ongoing pandemic infecting 219 million people as of 10/19/21, with a 3.6% mortality rate. Natural selection can generate favorable mutations with improved fitness advantages; however, the identified coronaviruses may be the tip of the iceberg, and potentially more fatal variants of concern (VOCs) may emerge over time. Understanding the patterns of emerging VOCs and forecasting mutations that may lead to gain of function or immune escape is urgently required. Here we developed PhyloTransformer, a Transformer-based discriminative model that engages a multi-head self-attention mechanism to model genetic mutations that may lead to viral reproductive advantage. In order to identify complex dependencies between the elements of each input sequence, PhyloTransformer utilizes advanced modeling techniques, including a novel Fast Attention Via positive Orthogonal Random features approach (FAVOR+) from Performer, and the Masked Language Model (MLM) from Bidirectional Encoder Representations from Transformers (BERT). PhyloTransformer was trained with 1,765,297 genetic sequences retrieved from the Global Initiative for Sharing All Influenza Data (GISAID) database. Firstly, we compared the prediction accuracy of novel mutations and novel combinations using extensive baseline models; we found that PhyloTransformer outperformed every baseline method with statistical significance. Secondly, we examined predictions of mutations in each nucleotide of the receptor binding motif (RBM), and we found our predictions were precise and accurate. Thirdly, we predicted modifications of N-glycosylation sites to identify mutations associated with altered glycosylation that may be favored during viral evolution. We anticipate that PhyloTransformer may guide proactive vaccine design for effective targeting of future SARS-CoV-2 variants.

CovTransformer: A transformer model for SARS-CoV-2 lineage frequency forecasting

TEMPO: A Transformer-Based Mutation Prediction Framework for SARS-CoV-2 Evolution

Predicting the Epidemics Trend of COVID-19 Using Epidemiological-Based Generative Adversarial Networks.

Integrating Transformer and GCN for COVID-19 Forecasting

CoViT: Real-time phylogenetics for the SARS-CoV-2 pandemic using Vision Transformers

PhyloTransformer: A Discriminative Model for Mutation Prediction Based on a Multi-head Self-attention Mechanism

ViTAL: Vision TrAnsformer based Low coverage SARS-CoV-2 lineage assignment

Spatial-Temporal Synchronous Graph Transformer network (STSGT) for COVID-19 forecasting

Interpreting Time Series Transformer Models and Sensitivity Analysis of Population Age Groups to COVID-19 Infections

Predicting sub-population specific viral evolution

Leveraging Wastewater Monitoring for COVID-19 Forecasting in the US: a Deep Learning study

Fine-Grained Forecasting of COVID-19 Trends at the County Level in the United States

High-resolution short-term prediction of the COVID-19 epidemic based on spatial-temporal model modified by historical meteorological data

COVINet: a deep learning-based and interpretable prediction model for the county-wise trajectories of COVID-19 in the United States

PhyloTransformer: A Self-supervised Discriminative Model for Mutation Prediction Based on a Multi-head Self-attention Mechanism

Running Ahead of Evolution - AI Based Simulation for Predicting Future High-Risk SARS-CoV-2 Variants

Evolution-guided Large Language Model is a Predictor of Virus Mutation Trends

Long-term prediction for temporal propagation of seasonal influenza using Transformer-based model

Deep COVID-19 Forecasting for Multiple States with Data Augmentation

RCoV19: A One-stop Hub for SARS-CoV-2 Genome Data Integration, Variant Monitoring, and Risk Pre-warning