Abstract:Introduction:: More recent self-supervised deep language models, such as Bidirectional Encoder Representations from Transformers (BERT), have performed the best on some language tasks by contextualizing word embeddings for a better dynamic representation. Their proteinspecific versions, such as ProtBERT, generated dynamic protein sequence embeddings, which resulted in better performance for several bioinformatics tasks. Besides, a number of different protein post-translational modifications are prominent in cellular tasks such as development and differentiation. The current biological experiments can detect these modifications, but within a longer duration and with a significant cost. Methods:: In this paper, to comprehend the accompanying biological processes concisely and more rapidly, we propose DEEPPTM to predict protein post-translational modification (PTM) sites from protein sequences more efficiently. Different than the current methods, DEEPPTM enhances the modification prediction performance by integrating specialized ProtBERT-based protein embeddings with attention-based vision transformers (ViT), and reveals the associations between different modification types and protein sequence content. Additionally, it can infer several different modifications over different species. Results:: Human and mouse ROC AUCs for predicting Succinylation modifications were 0.988 and 0.965 respectively, once 10-fold cross-validation is applied. Similarly, we have obtained 0.982, 0.955, and 0.953 ROC AUC scores on inferring ubiquitination, crotonylation, and glycation sites, respectively. According to detailed computational experiments, DEEPPTM lessens the time spent in laboratory experiments while outperforming the competing methods as well as baselines on inferring all 4 modification sites. In our case, attention-based deep learning methods such as vision transformers look more favorable to learning from ProtBERT features than more traditional deep learning and machine learning techniques. Conclusion:: Additionally, the protein-specific ProtBERT model is more effective than the original BERT embeddings for PTM prediction tasks. Our code and datasets can be found at https://github.com/seferlab/deepptm.

Improving protein succinylation sites prediction using embeddings from protein language model

Detecting Succinylation Sites from Protein Sequences Using Ensemble Support Vector Machine

SSKM_Succ: A Novel Succinylation Sites Prediction Method Incorporating K-Means Clustering with a New Semi-Supervised Learning Algorithm.

Accurate in Silico Identification of Protein Succinylation Sites Using an Iterative Semi-Supervised Learning Technique.

SuccSPred2.0: A Two-Step Model to Predict Succinylation Sites Based on Multifeature Fusion and Selection Algorithm.

HybridSucc: A Hybrid-learning Architecture for General and Species-specific Succinylation Site Prediction

SuccinSite: a computational tool for the prediction of protein succinylation sites by exploiting the amino acid patterns and properties.

SuccSPred: Succinylation Sites Prediction Using Fused Feature Representation and Ranking Method

pLMSNOSite: an ensemble-based approach for predicting protein S-nitrosylation sites by integrating supervised word embedding and embedding from pre-trained protein language model

LSTMCNNsucc: A Bidirectional LSTM and CNN-Based Deep Learning Method for Predicting Lysine Succinylation Sites

Isuc-Pseaac: Predicting Lysine Succinylation in Proteins by Incorporating Peptide Position-Specific Propensity.

LMCrot: An enhanced protein crotonylation site predictor by leveraging an interpretable window-level embedding from a transformer-based protein language model

SuccFind: a Novel Succinylation Sites Online Prediction Tool Via Enhanced Characteristic Strategy.

Deep_KsuccSite: A novel deep learning method for the identification of lysine succinylation sites

Extensive comparison of protein sequence-based bioinformatics applications for predicting lysine succinylation sites: a comparative review

pSuc-FFSEA: Predicting Lysine Succinylation Sites in Proteins Based on Feature Fusion and Stacking Ensemble Algorithm

SulSite-GTB: Identification of Protein S-sulfenylation Sites by Fusing Multiple Feature Information and Gradient Tree Boosting

pSuc-PseRat: Predicting Lysine Succinylation in Proteins by Exploiting the Ratios of Sequence Coupling and Properties.

Prediction of Acetylation and Succinylation in Proteins Based on Multilabel Learning RankSVM

DeepPTM: Protein Post-translational Modification Prediction from Protein Sequences by Combining Deep Protein Language Model with Vision Transformers

Leveraging protein language model embeddings and logistic regression for efficient and accurate in-silico acidophilic proteins classification