Abstract:Abstract Background One common task in Computational Biology is the prediction of aspects of protein function and structure from their amino acid sequence. For 26 years, most state-of-the-art approaches toward this end have been marrying machine learning and evolutionary information. The retrieval of related proteins from ever growing sequence databases is becoming so time-consuming that the analysis of entire proteomes becomes challenging. On top, evolutionary information is less powerful for small families, e.g. for proteins from the Dark Proteome . Results We introduce a novel way to represent protein sequences as continuous vectors ( embeddings ) by using the deep bi-directional model ELMo taken from natural language processing (NLP). The model has effectively captured the biophysical properties of protein sequences from unlabeled big data (UniRef50). After training, this knowledge is transferred to single protein sequences by predicting relevant sequence features. We refer to these new embeddings as SeqVec ( Seq uence-to- Vec tor) and demonstrate their effectiveness by training simple convolutional neural networks on existing data sets for two completely different prediction tasks. At the per-residue level, we significantly improved secondary structure (for NetSurfP-2.0 data set: Q3=79%±1, Q8=68%±1) and disorder predictions (MCC=0.59±0.03) over methods not using evolutionary information. At the per-protein level, we predicted subcellular localization in ten classes (for DeepLoc data set: Q10=68%±1) and distinguished membrane-bound from water-soluble proteins (Q2= 87%±1). All results built upon the embeddings gained from the new tool SeqVec neither explicitly nor implicitly using evolutionary information. Nevertheless, it improved over some methods using such information. Where the lightning-fast HHblits needed on average about two minutes to generate the evolutionary information for a target protein, SeqVec created the vector representation on average in 0.03 seconds. Conclusion We have shown that transfer learning can be used to capture biochemical or biophysical properties of protein sequences from large unlabeled sequence databases. The effectiveness of the proposed approach was showcased for different prediction tasks using only single protein sequences. SeqVec embeddings enable predictions that outperform even some methods using evolutionary information. Thus, they prove to condense the underlying principles of protein sequences. This might be the first step towards competitive predictions based only on single protein sequences. Availability SeqVec: https://github.com/mheinzinger/SeqVec Prediction server: https://embed.protein.properties

Tree visualizations of protein sequence embedding space enable improved functional clustering of diverse protein superfamilies

Exploring large protein sequence space through homology- and representation-based hierarchical clustering

When Protein Structure Embedding Meets Large Language Models

ACCURATE CLASSIFICATION OF PROTEIN STRUCTURAL FAMILIES USING COHERENT SUBGRAPH ANALYSIS

Detecting remote evolutionary relationships among proteins by large-scale semantic embedding

CProtMEDIAS: Clustering of Amino Acid Sequences Encoded by Gene Families by MErging and DIgitizing Aligned Sequences

Neural Embeddings for Protein Graphs

Modeling the language of life – Deep Learning Protein Sequences

EmbeddingTree: Hierarchical Exploration of Entity Features in Embedding

Functional Neighbors: Inferring Relationships Between Non-Homologous Protein Families Using Family-Specific Packing Motifs

Clustering protein functional families at large scale with hierarchical approaches

Beyond ESM2: Graph-Enhanced Protein Sequence Modeling with Efficient Clustering

Distributed Representation of Protein Sequence Based on Multi-Alignment Results

Embedding-based alignment: combining protein language models with dynamic programming alignment to detect structural similarities in the twilight-zone

Deep Learning Methods for Protein Family Classification on PDB Sequencing Data

Unsupervised Learning of Phylogenetic Trees via Split-Weight Embedding

A Rapid Method for Characterization of Protein Relatedness Using Feature Vectors

FoldExplorer: Fast and Accurate Protein Structure Search with Sequence-Enhanced Graph Embedding

Uncovering new families and folds in the natural protein universe

Comparing Graph Representations of Protein Structure for Mining Family-Specific Residue-Based Packing Motifs

A Discriminative Method for Protein Remote Homology Detection and Fold Recognition Combining Top-n-grams and Latent Semantic Analysis