Abstract:Large Language Models (LLMs) have achieved remarkable performance in multiple Natural Language Processing (NLP) tasks. Under the premise that protein sequences constitute the protein language, Protein Language Models(PLMs) have advanced the field of protein engineering. However, as of now, unlike LLMs in NLP, PLMs cannot handle the protein understanding task and the protein generation task simultaneously in the Protein Language Processing (PLP) field. This prompts us to delineate the inherent limitations in current PLMs: (i) the lack of natural language capabilities, (ii) insufficient instruction understanding, and (iii) high training resource demands. To address these challenges, we introduce a training framework to transform any general LLM into a PLM capable of handling multiple PLP tasks. To improve training efficiency, we propose Protein Vocabulary Pruning (PVP) for general LLMs. We construct a multi-task instruction dataset containing 13 million samples with superfamily information, facilitating better modeling of protein sequence-function landscapes. Through these methods, we develop the ProLLaMA model, the first known PLM to handle multiple PLP tasks simultaneously. Experiments show that ProLLaMA achieves state-of-the-art results in the unconditional protein sequence generation task. In the controllable protein sequence generation task, ProLLaMA can design novel proteins with desired functionalities. As for the protein understanding task, ProLLaMA achieves a 62\% exact match rate in superfamily prediction. Codes, model weights, and datasets are available at \url{<a class="link-external link-https" href="https://github.com/PKU-YuanGroup/ProLLaMA" rel="external noopener nofollow">this https URL</a>} and \url{<a class="link-external link-https" href="https://huggingface.co/GreatCaptainNemo" rel="external noopener nofollow">this https URL</a>}.

Does protein pretrained language model facilitate the prediction of protein–ligand interaction?

PLM-interact: extending protein language models to predict protein-protein interactions

Interpretable improving prediction performance of general protein language model by domain-adaptive pretraining on DNA-binding protein

Multi-PLI: interpretable multi‐task deep learning model for unifying protein–ligand interaction datasets

Multimodal Protein-Ligand Contrastive Pretraining for Effective and Efficient Drug Discovery

ProLLM: Protein Chain-of-Thoughts Enhanced LLM for Protein-Protein Interaction Prediction

PreDBP-PLMs: Prediction of DNA-binding proteins based on pre-trained protein language models and convolutional neural networks

ProtT3: Protein-to-Text Generation for Text-based Protein Understanding

A Brief Review of Protein–ligand Interaction Prediction

Exploring evolution-aware & -free protein language models as protein function predictors

Natural Language Processing Methods for the Study of Protein-Ligand Interactions

Protein language model-embedded geometric graphs power inter-protein contact prediction

PLMC: Language Model of Protein Sequences Enhances Protein Crystallization Prediction

Protein language model embedded geometric graphs power inter-protein contact prediction

InstructPLM: Aligning Protein Language Models to Follow Protein Structure Instructions

Efficient Inference, Training, and Fine-tuning of Protein Language Models

Harnessing Pre-trained Models for Accurate Prediction of Protein-Ligand Binding Affinity

Open-Source Protein Language Models for Function Prediction and Protein Design

ProLLaMA: A Protein Language Model for Multi-Task Protein Language Processing

From PSSM to Pre-Trained Language Models