Abstract:Abstract Background One common task in Computational Biology is the prediction of aspects of protein function and structure from their amino acid sequence. For 26 years, most state-of-the-art approaches toward this end have been marrying machine learning and evolutionary information. The retrieval of related proteins from ever growing sequence databases is becoming so time-consuming that the analysis of entire proteomes becomes challenging. On top, evolutionary information is less powerful for small families, e.g. for proteins from the Dark Proteome . Results We introduce a novel way to represent protein sequences as continuous vectors ( embeddings ) by using the deep bi-directional model ELMo taken from natural language processing (NLP). The model has effectively captured the biophysical properties of protein sequences from unlabeled big data (UniRef50). After training, this knowledge is transferred to single protein sequences by predicting relevant sequence features. We refer to these new embeddings as SeqVec ( Seq uence-to- Vec tor) and demonstrate their effectiveness by training simple convolutional neural networks on existing data sets for two completely different prediction tasks. At the per-residue level, we significantly improved secondary structure (for NetSurfP-2.0 data set: Q3=79%±1, Q8=68%±1) and disorder predictions (MCC=0.59±0.03) over methods not using evolutionary information. At the per-protein level, we predicted subcellular localization in ten classes (for DeepLoc data set: Q10=68%±1) and distinguished membrane-bound from water-soluble proteins (Q2= 87%±1). All results built upon the embeddings gained from the new tool SeqVec neither explicitly nor implicitly using evolutionary information. Nevertheless, it improved over some methods using such information. Where the lightning-fast HHblits needed on average about two minutes to generate the evolutionary information for a target protein, SeqVec created the vector representation on average in 0.03 seconds. Conclusion We have shown that transfer learning can be used to capture biochemical or biophysical properties of protein sequences from large unlabeled sequence databases. The effectiveness of the proposed approach was showcased for different prediction tasks using only single protein sequences. SeqVec embeddings enable predictions that outperform even some methods using evolutionary information. Thus, they prove to condense the underlying principles of protein sequences. This might be the first step towards competitive predictions based only on single protein sequences. Availability SeqVec: https://github.com/mheinzinger/SeqVec Prediction server: https://embed.protein.properties

A Unified Deep Biological Sequence Representation Learning with Pretrained Encoder Decoder Model

Unified Deep Learning Architecture for Modeling Biology Sequence.

Learning Complete Protein Representation by Deep Coupling of Sequence and Structure

Enhancing the Prediction of Protein Coding Regions in Biological Sequence Via a Deep Learning Framework with Hybrid Encoding

Sequence Representation Approaches for Sequence-Based Protein Prediction Tasks That Use Deep Learning.

Modeling the language of life – Deep Learning Protein Sequences

BioSequence2Vec: Efficient Embedding Generation For Biological Sequences

A High Efficient Biological Language Model for Predicting Protein–Protein Interactions

Protein-DNA Binding Residues Prediction Using a Deep Learning Model with Hierarchical Feature Extraction

Deciphering the Language of Protein-DNA Interactions: A Deep Learning Approach Combining Contextual Embeddings and Multi-Scale Sequence Modeling

An Integration of Deep Learning with Feature Embedding for Protein–protein Interaction Prediction

DeepBIO: an automated and interpretable deep-learning platform for high-throughput biological sequence prediction, functional annotation and visualization analysis

Bgfe: A Deep Learning Model For Ncrna-Protein Interaction Predictions Based On Improved Sequence Information

RNAincoder: a deep learning-based encoder for RNA and RNA-associated interaction

Combining High Speed ELM Learning with a Deep Convolutional Neural Network Feature Encoding for Predicting Protein-RNA Interactions.

Autobioseqpy: A Deep Learning Tool For The Classification Of Biological Sequences

Sequence2Vec: a Novel Embedding Approach for Modeling Transcription Factor Binding Affinity Landscape

A Systematic Study of Joint Representation Learning on Protein Sequences and Structures

Efficient Framework For Predicting Ncrna-Protein Interactions Based On Sequence Information By Deep Learning

D2VCB: A Hybrid Deep Neural Network for the Prediction of in-vivo Protein-DNA Binding from Combined DNA Sequence

ProtVec: A Continuous Distributed Representation of Biological Sequences