A Protein-Protein Interaction Extraction Approach Based on Large Pre-trained Language Model and Adversarial Training.

Zhan Tang,Xuchao Guo,Zhao Bai,Lei Diao,Shuhan Lu,Lin Li

DOI: https://doi.org/10.3837/tiis.2022.03.002

2022-01-01

KSII Transactions on Internet and Information Systems

Abstract:Adversarial embedding Experiments achieved utilize approaches extraction [1] [2] [3] [4]

What problem does this paper attempt to address?

Deep Learning Frameworks for Protein–protein Interaction Prediction

Xiaotian Hu,Cong Feng,Tianyi Ling,Ming Chen

DOI: https://doi.org/10.1016/j.csbj.2022.06.025

IF: 6.155

2022-01-01

Computational and Structural Biotechnology Journal

Abstract:Protein-protein interactions (PPIs) play key roles in a broad range of biological processes. The disorder of PPIs often causes various physical and mental diseases, which makes PPIs become the focus of the research on disease mechanism and clinical treatment. Since a large number of PPIs have been identified by in vivo and in vitro experimental techniques, the increasing scale of PPI data with the inherent complexity of interacting mechanisms has encouraged a growing use of computational methods to predict PPIs. Until recently, deep learning plays an increasingly important role in the machine learning field due to its remarkable non-linear transformation ability. In this article, we aim to present readers with a comprehensive introduction of deep learning in PPI prediction, including the diverse learning architectures, benchmarks and extended applications.(c) 2022 The Authors. Published by Elsevier B.V. on behalf of Research Network of Computational and Structural Biotechnology. This is an open access article under the CC BY license (http://creativecommons. org/licenses/by/4.0/).
Modeling Protein Using Large-scale Pretrain Language Model

Yijia Xiao,Jiezhong Qiu,Ziang Li,Chang-Yu Hsieh,Jie Tang

DOI: https://doi.org/10.48550/arXiv.2108.07435

2021-12-08

Abstract:Protein is linked to almost every life process. Therefore, analyzing the biological structure and property of protein sequences is critical to the exploration of life, as well as disease detection and drug discovery. Traditional protein analysis methods tend to be labor-intensive and time-consuming. The emergence of deep learning models makes modeling data patterns in large quantities of data possible. Interdisciplinary researchers have begun to leverage deep learning methods to model large biological datasets, e.g. using long short-term memory and convolutional neural network for protein sequence classification. After millions of years of evolution, evolutionary information is encoded in protein sequences. Inspired by the similarity between natural language and protein sequences, we use large-scale language models to model evolutionary-scale protein sequences, encoding protein biology information in representation. Significant improvements are observed in both token-level and sequence-level tasks, demonstrating that our large-scale model can accurately capture evolution information from pretraining on evolutionary-scale individual sequences. Our code and model are available at <a class="link-external link-https" href="https://github.com/THUDM/ProteinLM" rel="external noopener nofollow">this https URL</a>.

Machine Learning,Computation and Language,Biomolecules
ProtChatGPT: Towards Understanding Proteins with Large Language Models

Chao Wang,Hehe Fan,Ruijie Quan,Yi Yang

2024-02-15

Abstract:Protein research is crucial in various fundamental disciplines, but understanding their intricate structure-function relationships remains challenging. Recent Large Language Models (LLMs) have made significant strides in comprehending task-specific knowledge, suggesting the potential for ChatGPT-like systems specialized in protein to facilitate basic research. In this work, we introduce ProtChatGPT, which aims at learning and understanding protein structures via natural languages. ProtChatGPT enables users to upload proteins, ask questions, and engage in interactive conversations to produce comprehensive answers. The system comprises protein encoders, a Protein-Language Pertaining Transformer (PLP-former), a projection adapter, and an LLM. The protein first undergoes protein encoders and PLP-former to produce protein embeddings, which are then projected by the adapter to conform with the LLM. The LLM finally combines user questions with projected embeddings to generate informative answers. Experiments show that ProtChatGPT can produce promising responses to proteins and their corresponding questions. We hope that ProtChatGPT could form the basis for further exploration and application in protein research. Code and our pre-trained model will be publicly available.

Computational Engineering, Finance, and Science,Artificial Intelligence,Biomolecules
Protein-protein Interaction Extraction Based on Self-Training

Xiaoyan Zhu

2012-01-01

Abstract:Protein-protein interaction(PPI) extraction based on supervised machine learning still faces the problem that a limited labeled dataset does not saturate the learning method.This study first presents a rich feature vector,including syntactic features,lexical features,and part-of-speech(POS) tag features and then analyzes the inconsistencies of data distributions between different datasets,with a data enlargement algorithm based on self-training.This algorithm chooses the most confident instances from the unlabeled dataset into the target dataset to improve the learning method.Tests show that the PPI extraction approach considerably improves the extraction efficiency on five public PPI datasets.
ProtLLM: An Interleaved Protein-Language LLM with Protein-as-Word Pre-Training

Le Zhuo,Zewen Chi,Minghao Xu,Heyan Huang,Heqi Zheng,Conghui He,Xian-Ling Mao,Wentao Zhang

2024-02-28

Abstract:We propose ProtLLM, a versatile cross-modal large language model (LLM) for both protein-centric and protein-language tasks. ProtLLM features a unique dynamic protein mounting mechanism, enabling it to handle complex inputs where the natural language text is interspersed with an arbitrary number of proteins. Besides, we propose the protein-as-word language modeling approach to train ProtLLM. By developing a specialized protein vocabulary, we equip the model with the capability to predict not just natural language but also proteins from a vast pool of candidates. Additionally, we construct a large-scale interleaved protein-text dataset, named InterPT, for pre-training. This dataset comprehensively encompasses both (1) structured data sources like protein annotations and (2) unstructured data sources like biological research papers, thereby endowing ProtLLM with crucial knowledge for understanding proteins. We evaluate ProtLLM on classic supervised protein-centric tasks and explore its novel protein-language applications. Experimental results demonstrate that ProtLLM not only achieves superior performance against protein-specialized baselines on protein-centric tasks but also induces zero-shot and in-context learning capabilities on protein-language tasks.

Biomolecules,Artificial Intelligence,Computation and Language,Machine Learning
SPRoBERTa: protein embedding learning with local fragment modeling

Lijun Wu,Chengcan Yin,Jinhua Zhu,Zhen Wu,Liang He,Yingce Xia,Shufang Xie,Tao Qin,Tie-Yan Liu

DOI: https://doi.org/10.1093/bib/bbac401

IF: 9.5

2022-09-22

Briefings in Bioinformatics

Abstract:Abstract Well understanding protein function and structure in computational biology helps in the understanding of human beings. To face the limited proteins that are annotated structurally and functionally, the scientific community embraces the self-supervised pre-training methods from large amounts of unlabeled protein sequences for protein embedding learning. However, the protein is usually represented by individual amino acids with limited vocabulary size (e.g. 20 type proteins), without considering the strong local semantics existing in protein sequences. In this work, we propose a novel pre-training modeling approach SPRoBERTa. We first present an unsupervised protein tokenizer to learn protein representations with local fragment pattern. Then, a novel framework for deep pre-training model is introduced to learn protein embeddings. After pre-training, our method can be easily fine-tuned for different protein tasks, including amino acid-level prediction task (e.g. secondary structure prediction), amino acid pair-level prediction task (e.g. contact prediction) and also protein-level prediction task (remote homology prediction, protein function prediction). Experiments show that our approach achieves significant improvements in all tasks and outperforms the previous methods. We also provide detailed ablation studies and analysis for our protein tokenizer and training framework.

biochemical research methods,mathematical & computational biology
Automatic Extraction of Protein Interaction in Literature

Peilei Liu,Ting Wang

DOI: https://doi.org/10.48550/arXiv.1406.1953

2014-06-08

Computation and Language

Abstract:Protein-protein interaction extraction is the key precondition of the construction of protein knowledge network, and it is very important for the research in the biomedicine. This paper extracted directional protein-protein interaction from the biological text, using the SVM-based method. Experiments were evaluated on the LLL05 corpus with good results. The results show that dependency features are import for the protein-protein interaction extraction and features related to the interaction word are effective for the interaction direction judgment. At last, we analyzed the effects of different features and planed for the next step.
ProteinAligner: A Multi-modal Pretraining Framework for Protein Foundation Models

Li Zhang,Han Guo,Leah V Schaffer,Young Su Ko,Digvijay Singh,Hamid Rahmani,Danielle Grotjahn,Elizabeth Villa,Michael Gilson,Wei Wang,Trey Ideker,Eric Xing,Pengtao Xie

DOI: https://doi.org/10.1101/2024.10.06.616870

2024-10-06

Abstract:Protein foundation models, particularly protein language models, have demonstrated strong success in learning meaningful representations of proteins using transformer architectures pretrained on large-scale protein datasets with self-supervised learning. These representations have been highly effective for downstream tasks such as predicting protein functions and properties. However, most current protein foundation models focus on pretraining with amino acid sequences, often neglecting additional modalities like protein structures and related literature, both of which provide valuable insights. To address this gap, we propose a multi-modal pretraining approach that integrates three key modalities - protein sequences, structures, and literature text. In our framework, the protein sequence modality serves as the anchor, with the other two modalities aligned to it, enhancing the model's capacity to capture more comprehensive protein information. ProteinAligner outperformed state-of-the-art protein foundation models in predicting protein functions and properties across diverse downstream tasks.

Bioinformatics
Protein–protein interaction and site prediction using transfer learning

Tuoyu Liu,Han Gao,Xiaopu Ren,Guoshun Xu,Bo Liu,Ningfeng Wu,Huiying Luo,Yuan Wang,Tao Tu,Bin Yao,Feifei Guan,Yue Teng,Huoqing Huang,Jian Tian

DOI: https://doi.org/10.1093/bib/bbad376

IF: 9.5

2023-10-24

Briefings in Bioinformatics

Abstract:The advanced language models have enabled us to recognize protein–protein interactions (PPIs) and interaction sites using protein sequences or structures. Here, we trained the MindSpore ProteinBERT (MP-BERT) model, a Bidirectional Encoder Representation from Transformers, using protein pairs as inputs, making it suitable for identifying PPIs and their respective interaction sites. The pretrained model (MP-BERT) was fine-tuned as MPB-PPI (MP-BERT on PPI) and demonstrated its superiority over the state-of-the-art models on diverse benchmark datasets for predicting PPIs. Moreover, the model's capability to recognize PPIs among various organisms was evaluated on multiple organisms. An amalgamated organism model was designed, exhibiting a high level of generalization across the majority of organisms and attaining an accuracy of 92.65%. The model was also customized to predict interaction site propensity by fine-tuning it with PPI site data as MPB-PPISP. Our method facilitates the prediction of both PPIs and their interaction sites, thereby illustrating the potency of transfer learning in dealing with the protein pair task.

biochemical research methods,mathematical & computational biology
Uncovering protein interaction in abstracts and text using a novel linear model and word proximity networks

Alaa Abi-Haidar,Jasleen Kaur,Ana G. Maguitman,Predrag Radivojac,Andreas Retchsteiner,Karin Verspoor,Zhiping Wang,Luis M. Rocha

DOI: https://doi.org/10.1186/gb-2008-9-s2-s11

2008-12-05

Abstract:We participated in three of the protein-protein interaction subtasks of the Second BioCreative Challenge: classification of abstracts relevant for protein-protein interaction (IAS), discovery of protein pairs (IPS) and text passages characterizing protein interaction (ISS) in full text documents. We approached the abstract classification task with a novel, lightweight linear model inspired by spam-detection techniques, as well as an uncertainty-based integration scheme. We also used a Support Vector Machine and the Singular Value Decomposition on the same features for comparison purposes. Our approach to the full text subtasks (protein pair and passage identification) includes a feature expansion method based on word-proximity networks. Our approach to the abstract classification task (IAS) was among the top submissions for this task in terms of the measures of performance used in the challenge evaluation (accuracy, F-score and AUC). We also report on a web-tool we produced using our approach: the Protein Interaction Abstract Relevance Evaluator (PIARE). Our approach to the full text tasks resulted in one of the highest recall rates as well as mean reciprocal rank of correct passages. Our approach to abstract classification shows that a simple linear model, using relatively few features, is capable of generalizing and uncovering the conceptual nature of protein-protein interaction from the bibliome. Since the novel approach is based on a very lightweight linear model, it can be easily ported and applied to similar problems. In full text problems, the expansion of word features with word-proximity networks is shown to be useful, though the need for some improvements is discussed.

Information Retrieval,Machine Learning
Protein Representation Learning via Knowledge Enhanced Primary Structure Modeling

Hong-Yu Zhou,Yunxiang Fu,Zhicheng Zhang,Cheng Bian,Yizhou Yu

2023-02-15

Abstract:Protein representation learning has primarily benefited from the remarkable development of language models (LMs). Accordingly, pre-trained protein models also suffer from a problem in LMs: a lack of factual knowledge. The recent solution models the relationships between protein and associated knowledge terms as the knowledge encoding objective. However, it fails to explore the relationships at a more granular level, i.e., the token level. To mitigate this, we propose Knowledge-exploited Auto-encoder for Protein (KeAP), which performs token-level knowledge graph exploration for protein representation learning. In practice, non-masked amino acids iteratively query the associated knowledge tokens to extract and integrate helpful information for restoring masked amino acids via attention. We show that KeAP can consistently outperform the previous counterpart on 9 representative downstream applications, sometimes surpassing it by large margins. These results suggest that KeAP provides an alternative yet effective way to perform knowledge enhanced protein representation learning.

Machine Learning,Computation and Language,Multimedia
Deep learning with feature embedding for compound-protein interaction prediction

Fangping Wan,Jianyang (Michael) Zeng

DOI: https://doi.org/10.1101/086033

2016-11-07

Abstract:Abstract Accurately identifying compound-protein interactions in silico can deepen our understanding of the mechanisms of drug action and significantly facilitate the drug discovery and development process. Traditional similarity-based computational models for compound-protein interaction prediction rarely exploit the latent features from current available large-scale unlabelled compound and protein data, and often limit their usage on relatively small-scale datasets. We propose a new scheme that combines feature embedding (a technique of representation learning) with deep learning for predicting compound-protein interactions. Our method automatically learns the low-dimensional implicit but expressive features for compounds and proteins from the massive amount of unlabelled data. Combining effective feature embedding with powerful deep learning techniques, our method provides a general computational pipeline for accurate compound-protein interaction prediction, even when the interaction knowledge of compounds and proteins is entirely unknown. Evaluations on current large-scale databases of the measured compound-protein affinities, such as ChEMBL and BindingDB, as well as known drug-target interactions from DrugBank have demonstrated the superior prediction performance of our method, and suggested that it can offer a useful tool for drug development and drug repositioning.
Pre-Training Protein Bi-level Representation Through Span Mask Strategy On 3D Protein Chains

Jiale Zhao,Wanru Zhuang,Jia Song,Yaqi Li,Shuqi Lu

2024-06-03

Abstract:In recent years, there has been a surge in the development of 3D structure-based pre-trained protein models, representing a significant advancement over pre-trained protein language models in various downstream tasks. However, most existing structure-based pre-trained models primarily focus on the residue level, i.e., alpha carbon atoms, while ignoring other atoms like side chain atoms. We argue that modeling proteins at both residue and atom levels is important since the side chain atoms can also be crucial for numerous downstream tasks, for example, molecular docking. Nevertheless, we find that naively combining residue and atom information during pre-training typically fails. We identify a key reason is the information leakage caused by the inclusion of atom structure in the input, which renders residue-level pre-training tasks trivial and results in insufficiently expressive residue representations. To address this issue, we introduce a span mask pre-training strategy on 3D protein chains to learn meaningful representations of both residues and atoms. This leads to a simple yet effective approach to learning protein representation suitable for diverse downstream tasks. Extensive experimental results on binding site prediction and function prediction tasks demonstrate our proposed pre-training approach significantly outperforms other methods. Our code will be made public.

Machine Learning,Artificial Intelligence,Biomolecules
ProteinCLIP: enhancing protein language models with natural language

Kevin E Wu,Howard Chang,James Zou

DOI: https://doi.org/10.1101/2024.05.14.594226

2024-05-17

Abstract:Language models have enabled a new era of biological sequence modeling. However, extracting meaningful sequence-level embeddings from these models remains challenging. In this work, we introduce ProteinCLIP, which applies contrastive learning between a protein's amino acid sequence and curated text describing its function. ProteinCLIP thus learns to take a pre-trained protein language model's sequence embedding and refines it produce a function-centric embedding. We show that this embedding space yields sequence representations that enable state-of-the-art performance across a variety of important yet challenging tasks in the study of proteins -- from predicting protein protein interactions to accurately detecting homologous proteins despite low sequence similarity. More broadly, ProteinCLIP demonstrates the effectiveness of multi-modal learning in biological contexts, and how such strategies can help isolate key signals from large models and further improve their utility.

Bioinformatics
OntoProtein: Protein Pretraining With Gene Ontology Embedding

Ningyu Zhang,Zhen Bi,Xiaozhuan Liang,Siyuan Cheng,Haosen Hong,Shumin Deng,Jiazhang Lian,Qiang Zhang,Huajun Chen

2022-06-04

Abstract:Self-supervised protein language models have proved their effectiveness in learning the proteins representations. With the increasing computational power, current protein language models pre-trained with millions of diverse sequences can advance the parameter scale from million-level to billion-level and achieve remarkable improvement. However, those prevailing approaches rarely consider incorporating knowledge graphs (KGs), which can provide rich structured knowledge facts for better protein representations. We argue that informative biology knowledge in KGs can enhance protein representation with external knowledge. In this work, we propose OntoProtein, the first general framework that makes use of structure in GO (Gene Ontology) into protein pre-training models. We construct a novel large-scale knowledge graph that consists of GO and its related proteins, and gene annotation texts or protein sequences describe all nodes in the graph. We propose novel contrastive learning with knowledge-aware negative sampling to jointly optimize the knowledge graph and protein embedding during pre-training. Experimental results show that OntoProtein can surpass state-of-the-art methods with pre-trained protein language models in TAPE benchmark and yield better performance compared with baselines in protein-protein interaction and protein function prediction. Code and datasets are available in <a class="link-external link-https" href="https://github.com/zjunlp/OntoProtein" rel="external noopener nofollow">this https URL</a>.

Biomolecules,Artificial Intelligence,Computation and Language,Information Retrieval,Machine Learning
PETA: Evaluating the Impact of Protein Transfer Learning with Sub-word Tokenization on Downstream Applications

Yang Tan,Mingchen Li,Pan Tan,Ziyi Zhou,Huiqun Yu,Guisheng Fan,Liang Hong

2023-10-26

Abstract:Large protein language models are adept at capturing the underlying evolutionary information in primary structures, offering significant practical value for protein engineering. Compared to natural language models, protein amino acid sequences have a smaller data volume and a limited combinatorial space. Choosing an appropriate vocabulary size to optimize the pre-trained model is a pivotal issue. Moreover, despite the wealth of benchmarks and studies in the natural language community, there remains a lack of a comprehensive benchmark for systematically evaluating protein language model quality. Given these challenges, PETA trained language models with 14 different vocabulary sizes under three tokenization methods. It conducted thousands of tests on 33 diverse downstream datasets to assess the models' transfer learning capabilities, incorporating two classification heads and three random seeds to mitigate potential biases. Extensive experiments indicate that vocabulary sizes between 50 and 200 optimize the model, whereas sizes exceeding 800 detrimentally affect the model's representational performance. Our code, model weights and datasets are available at <a class="link-external link-https" href="https://github.com/ginnm/ProteinPretraining" rel="external noopener nofollow">this https URL</a>.

Computation and Language,Artificial Intelligence,Biomolecules
Deciphering the Language of Protein-DNA Interactions: A Deep Learning Approach Combining Contextual Embeddings and Multi-Scale Sequence Modeling

Yu-Chen Liu,Yi-Jing Lin,Yan-Yun Chang,Cheng-Che Chuang,Yu-Yen Ou

DOI: https://doi.org/10.1016/j.jmb.2024.168769

2024-08-28

Abstract:Deciphering the mechanisms governing protein-DNA interactions is crucial for understanding key cellular processes and disease pathways. In this work, we present a powerful deep learning approach that significantly advances the computational prediction of DNA-interacting residues from protein sequences. Our method leverages the rich contextual representations learned by pre-trained protein language models, such as ProtTrans, to capture intrinsic biochemical properties and sequence motifs indicative of DNA binding sites. We then integrate these contextual embeddings with a multi-window convolutional neural network architecture, which scans across the sequence at varying window sizes to effectively identify both local and global binding patterns. Comprehensive evaluation on curated benchmark datasets demonstrates the remarkable performance of our approach, achieving an area under the ROC curve (AUC) of 0.89 - a substantial improvement over previous state-of-the-art sequence-based predictors. This showcases the immense potential of pairing advanced representation learning and deep neural network designs for uncovering the complex syntax governing protein-DNA interactions directly from primary sequences. Our work not only provides a robust computational tool for characterizing DNA-binding mechanisms, but also highlights the transformative opportunities at the intersection of language modeling, deep learning, and protein sequence analysis. The publicly available code and data further facilitate broader adoption and continued development of these techniques for accelerating mechanistic insights into vital biological processes and disease pathways. In addition, the code and data for this work are available at https://github.com/B1607/DIRP.
A hybrid method for extraction of protein-protein interactions from literature

Weizhong Qian,Ungar Lyle,Zhiguang Qin,Chong Fu

DOI: https://doi.org/10.3772/j.issn.1006-6748.2011.01.006

2011-01-01

Abstract:In this work, a hybrid method is proposed to eliminate the limitations of traditional protein-protein interactions(PPIs) extraction methods, such as pattern learning and machine learning. Each sentence from the biomedical literature containing a protein pair describes a PPI which is predicted by first learning syntax patterns typical of PPIs from training corpus and then using their presence as features, along with bag-of-word features in a maximum entropy model. Tested on the BioCreAtIve corpus, the PPIs extraction method, which achieved a precision rate of 64%, recall rate of 60%, improved the performance in terms of F1 value by 11% compared with the component pure pattern-based and bag-of-word methods. The results on this test set were also compared with other three extraction methods and found to improve the performance remarkably. Copyright © by High Technology Letters Press.
Protein-Protein Interaction Prediction is Achievable with Large Language Models

Hallee,L.,Gleghorn,J. P.

DOI: https://doi.org/10.1101/2023.06.07.544109

2023-06-10

bioRxiv

Abstract:Predicting protein-protein interactions (PPIs) is vital for elucidating fundamental biology, designing peptide therapeutics, and for high-throughput protein annotation. This is particularly relevant in the current biotechnology landscape characterized by the proliferation of protein generative models, which necessitate a high-throughput and generalized PPI predictor for proteins regardless of conventional motifs or known biological functions. Our work addresses this need and provides strong evidence of the utility and reliability of protein language models (pLMs) in learning the PPI objective. We demonstrated that with the use of a sizable balanced dataset, pLMs achieve state-of-the-art performance metrics in PPI prediction on diverse proteins. To generate a dataset that allows for the approximation of these conditions, we implemented a novel synthetic data generation scheme to augment BIOGRID and Negatome datasets. The enhancement of these datasets was then used to fine-tune ProtBERT for PPI prediction to develop a model that we call SYNTERACT (SYNThetic data-driven protein-protein intERACtion Transformer). Our results are compelling, demonstrating 92% accuracy on validated positive and negative interacting pairs derived from 50 different organisms, all of which were excluded from the training phase. In addition to the high metrics, secondary analysis revealed that our synthetic negative data was able to successfully mimic actual negative samples, further reinforcing the integrity of synthetic data additions to PPI datasets. Another notable discovery was the ease in which previously existing PPI datasets could be predicted with simplistic features, calling into question if they can actually inform PPI prediction. We find that the subcellular compartment bias inherent to the compilation of these datasets is learnable with deep learning methods and demonstrate that our approach is not burdened by this disadvantage.
Protein-protein interaction extraction from bio-literature with compact features and data sampling strategy

Hongtao Zhang,Minlie Huang,Xiaoyan Zhu

DOI: https://doi.org/10.1109/BMEI.2011.6098714

2011-01-01

Abstract:A large number of protein-protein interactions (PPIs) have buried in massive biomedical articles published over the years. This leads to the development of automatic PPI extraction methods. However, existing methods based on supervised machine learning still face some challenges: (1) the feature space exploited in these methods is very sparse; and (2) the data used for training are imbalanced with respect to categories to be classified. In this paper, we first construct rich and compact features to alleviate the issue of feature sparseness. With these features, our method outperforms baselines by up to an F-score of 9.58% on the original AIMed corpus. Furthermore, we propose a data sampling strategy based on under-sampling to address the class imbalance problem. In order to re-balance data distribution, samples of the majority class are removed according to the prediction results iteratively. By this means, our method achieves a further 2.49% improvement in F-score on the original AIMed corpus.

A Protein-Protein Interaction Extraction Approach Based on Large Pre-trained Language Model and Adversarial Training.

Deep Learning Frameworks for Protein–protein Interaction Prediction

Modeling Protein Using Large-scale Pretrain Language Model

ProtChatGPT: Towards Understanding Proteins with Large Language Models

Protein-protein Interaction Extraction Based on Self-Training

ProtLLM: An Interleaved Protein-Language LLM with Protein-as-Word Pre-Training

SPRoBERTa: protein embedding learning with local fragment modeling

Automatic Extraction of Protein Interaction in Literature

ProteinAligner: A Multi-modal Pretraining Framework for Protein Foundation Models

Protein–protein interaction and site prediction using transfer learning

Uncovering protein interaction in abstracts and text using a novel linear model and word proximity networks

Protein Representation Learning via Knowledge Enhanced Primary Structure Modeling

Deep learning with feature embedding for compound-protein interaction prediction

Pre-Training Protein Bi-level Representation Through Span Mask Strategy On 3D Protein Chains

ProteinCLIP: enhancing protein language models with natural language

OntoProtein: Protein Pretraining With Gene Ontology Embedding

PETA: Evaluating the Impact of Protein Transfer Learning with Sub-word Tokenization on Downstream Applications

Deciphering the Language of Protein-DNA Interactions: A Deep Learning Approach Combining Contextual Embeddings and Multi-Scale Sequence Modeling

A hybrid method for extraction of protein-protein interactions from literature

Protein-Protein Interaction Prediction is Achievable with Large Language Models

Protein-protein interaction extraction from bio-literature with compact features and data sampling strategy