Abstract:Gene expression involves transcription and translation. Despite large datasets and increasingly powerful methods devoted to calculating genetic variants’ effects on transcription, discrepancy between messenger RNA and protein levels hinders the systematic interpretation of the regulatory effects of disease-associated variants. Accurate models of the sequence determinants of translation are needed to close this gap and to interpret disease-associated variants that act on translation. Here we present Translatomer, a multimodal transformer framework that predicts cell-type-specific translation from messenger RNA expression and gene sequence. We train the Translatomer on 33 tissues and cell lines, and show that the inclusion of sequence improves the prediction of ribosome profiling signal, indicating that the Translatomer captures sequence-dependent translational regulatory information. The Translatomer achieves accuracies of 0.72 to 0.80 for the de novo prediction of cell-type-specific ribosome profiling. We develop an in silico mutagenesis tool to estimate mutational effects on translation and demonstrate that variants associated with translation regulation are evolutionarily constrained, both in the human population and across species. In particular, we identify cell-type-specific translational regulatory mechanisms independent of the expression quantitative trait loci for 3,041 non-coding and synonymous variants associated with complex diseases, including Alzheimer’s disease, schizophrenia and congenital heart disease. The Translatomer accurately models the genetic underpinnings of translation, bridging the gap between messenger RNA and protein levels as well as providing valuable mechanistic insights for uninterpreted disease variants. A transformer-based approach called Translatomer is presented, which models cell-type-specific translation from messenger RNA expression and gene sequence, bridging the gap between messenger RNA and protein levels as well as providing a mechanistic insight into the genetic regulation of translation.

xTrimoGene: An Efficient and Scalable Representation Learner for Single-Cell RNA-Seq Data

TransfoRNA: Navigating the Uncertainties of Small RNA Annotation with an Adaptive Machine Learning Strategy

SCTrans: Multi-scale Scrna-Seq Sub-vector Completion Transformer for Gene-selective Cell Type Annotation

xTrimoPGLM: Unified 100B-Scale Pre-trained Transformer for Deciphering the Language of Protein

Deep Learning Prediction of Ribosome Profiling with Translatomer Reveals Translational Regulation and Interprets Disease Variants

Scaling Dense Representations for Single Cell with Transcriptome-Scale Context

Sctranssort: Transformers for Intelligent Annotation of Cell Types by Gene Embeddings.

A Unified Deep Learning Framework for Single-Cell ATAC-Seq Analysis Based on ProdDep Transformer Encoder

scGREAT: Transformer-Based Deep-Language Model for Gene Regulatory Network Inference from Single-Cell Transcriptomics

3D-Transformer: Molecular Representation with Transformer in 3D Space

STGRNS: an interpretable transformer-based method for inferring gene regulatory networks from single-cell transcriptomic data

Integrating T-cell Receptor and Transcriptome for Large-Scale Single-Cell Immune Profiling Analysis

White-Box Diffusion Transformer for single-cell RNA-seq generation

Translator: A Transfer Learning Approach to Facilitate Single-Cell ATAC-Seq Data Analysis from Reference Dataset

Single-Cell Multimodal Prediction via Transformers

The Nucleotide Transformer: Building and Evaluating Robust Foundation Models for Human Genomics

TransCluster: A Cell-Type Identification Method for single-cell RNA-Seq data using deep learning based on transformer

ProtGO: A Transformer based Fusion Model for accurately predicting Gene Ontology (GO) Terms from full scale Protein Sequences

CodonTransformer: a multispecies codon optimizer using context-aware neural networks

TransGEM: a Molecule Generation Model Based on Transformer with Gene Expression Data.