Protein language models guide directed antibody evolution

Arunima Singh

DOI: https://doi.org/10.1038/s41592-023-01924-w

IF: 48

2023-06-10

Nature Methods

Abstract:Nature Methods, Published online: 09 June 2023; doi:10.1038/s41592-023-01924-w Protein language models guide directed antibody evolution

biochemical research methods

What problem does this paper attempt to address?

Protein Language Models in Directed Evolution

Russell Maguire,Kotryna Bloznelyte,Fikayo Adepoju,Matthew Armean-Jones,Shafiat Dewan,Akash Gupta,Frances Patricia Jones,Preet Lalli,Anna Schooneveld,Sean Thompson,Ece Ebrahimi,Stella Fozzard,David Berman,Luca Rossoni,Will Addison,Ian Taylor

DOI: https://doi.org/10.1101/2024.08.20.608752

2024-08-20

Abstract:The dominant paradigms for integrating machine-learning into protein engineering are de novo protein design and guided directed evolution. Guiding directed evolution requires a model of protein fitness, but most models are only evaluated in silico on datasets comprising few mutations. Due to the limited number of mutations in these datasets, it is unclear how well these models can guide directed evolution efforts. We demonstrate in vitro how zero-shot and few-shot protein language models of fitness can be used to guide two rounds of directed evolution with simulated annealing. Our few-shot simulated annealing approach recommended enzyme variants with 1.62 x improved PET degradation over 72 h period, outperforming the top engineered variant from the literature, which was 1.40 x fitter than wild-type. In the second round, 240 in vitro examples were used for training, 32 homologous sequences were used for evolutionary context and 176 variants were evaluated for improved PET degradation, achieving a hit-rate of 39 % of variants fitter than wild-type.

Bioinformatics
Addressing the antibody germline bias and its effect on language models for improved antibody design

Tobias H Olsen,Iain H Moal,Charlotte M Deane

DOI: https://doi.org/10.1093/bioinformatics/btae618

IF: 5.8

2024-10-26

Bioinformatics

Abstract:Motivation: The versatile binding properties of antibodies have made them an extremely important class of biotherapeutics. However, therapeutic antibody development is a complex, expensive and time-consuming task, with the final antibody needing to not only have strong and specific binding, but also be minimally impacted by developability issues. The success of transformer-based language models in protein sequence space and the availability of vast amounts of antibody sequences, has led to the development of many antibody-specific language models to help guide antibody design. Antibody diversity primarily arises from V(D)J recombination, mutations within the CDRs, and/or from a few non-germline mutations outside the CDRs. Consequently, a significant portion of the variable domain of all natural antibody sequences remains germline. This affects the pre-training of antibody-specific language models, where this facet of the sequence data introduces a prevailing bias towards germline residues. This poses a challenge, as mutations away from the germline are often vital for generating specific and potent binding to a target, meaning that language models need be able to suggest key mutations away from germline. Results: In this study, we explore the implications of the germline bias, examining its impact on both general-protein and antibody-specific language models. We develop and train a series of new antibody-specific language models optimised for predicting non-germline residues. We then compare our final model, AbLang-2, with current models and show how it suggests a diverse set of valid mutations with high cumulative probability. Availability and implementation: AbLang-2 is trained on both unpaired and paired data, and is freely available at https://github.com/oxpig/AbLang2.git. Supplementary information: Supplementary data are available at Journal Name online.
Learning the protein language: Evolution, structure, and function

Tristan Bepler,Bonnie Berger

DOI: https://doi.org/10.1016/j.cels.2021.05.017

2021-06-16

Abstract:Language models have recently emerged as a powerful machine-learning approach for distilling information from massive protein sequence databases. From readily available sequence data alone, these models discover evolutionary, structural, and functional organization across protein space. Using language models, we can encode amino-acid sequences into distributed vector representations that capture their structural and functional properties, as well as evaluate the evolutionary fitness of sequence variants. We discuss recent advances in protein language modeling and their applications to downstream protein property prediction problems. We then consider how these models can be enriched with prior biological knowledge and introduce an approach for encoding protein structural knowledge into the learned representations. The knowledge distilled by these models allows us to improve downstream function prediction through transfer learning. Deep protein language models are revolutionizing protein biology. They suggest new ways to approach protein and therapeutic design. However, further developments are needed to encode strong biological priors into protein language models and to increase their accessibility to the broader community.
Knowledge-aware Reinforced Language Models for Protein Directed Evolution

Yuhao Wang,Qiang Zhang,Ming Qin,Xiang Zhuang,Xiaotong Li,Zhichen Gong,Zeyuan Wang,Yu Zhao,Jianhua Yao,Keyan Ding,Huajun Chen

2024-01-01

Abstract:Directed evolution, a cornerstone of protein optimization, is to harness natural mutational processes to enhance protein functionality. Existing Machine Learning-assisted Directed Evolution (MLDE) methodologies typically rely on data-driven strategies and often overlook the profound domain knowledge in biochemical fields. In this paper, we introduce a novel Knowledge-aware Reinforced Language Model (KnowRLM) for MLDE. An Amino Acid Knowledge Graph (AAKG) is constructed to represent the intricate biochemical relationships among amino acids. We further propose a Protein Language Model (PLM)-based policy network that iteratively samples mutants through preferential random walks on the AAKG using a dynamic sliding window mechanism. The novel mutants are actively sampled to fine-tune a fitness predictor as the reward model, providing feedback to the knowledge-aware policy. Finally, we optimize the whole system in an active learning approach that mimics biological settings in practice.KnowRLM stands out for its ability to utilize contextual amino acid information from knowledge graphs, thus attaining advantages from both statistical patterns of protein sequences and biochemical properties of amino acids.Extensive experiments demonstrate the superior performance of KnowRLM in more efficiently identifying high-fitness mutants compared to existing methods.
Learning the Language of Antibody Hypervariability

Rohit Singh,Chiho Im,Yu Qiu,Brian Mackness,Abhinav Gupta,Taylor Sorenson,Samuel Sledzieski,Lena Erlach,Maria Wendt,Yves Nanfack,Bryan D Bryson,Bonnie Berger

DOI: https://doi.org/10.1101/2023.04.26.538476

2024-05-29

Abstract:Protein language models (PLMs) based on machine learning have demonstrated impressive success in predicting protein structure and function. However, general-purpose ("foundational") PLMs have limited performance in predicting antibodies due to the latter's hypervariable regions, which do not conform to the evolutionary conservation principles that the models rely on. In this study, we propose a new transfer learning framework called AbMAP, which fine-tunes foundational models for antibody-sequence inputs by supervising on antibody structure and binding specificity examples. Our feature representations accurately predict an antibody's 3D structure, mutational effects on antigen binding, and paratope identification. AbMAP's scalability paves the way for large-scale analyses of human antibody repertoires. AbMAP representations of repertoires reveal a remarkable overlap across individuals, transcending the limits of sequence analyses. Our findings provide compelling evidence for the hypothesis that antibody repertoires of individuals tend to converge towards comparable structural and functional coverage. We validate AbMAP for antibody optimization, applying it to optimize a set of antibodies that bind to a SARS-CoV-2 peptide and obtaining 82% hit-rate and upto 22-fold increase in binding affinity. We anticipate AbMAP will accelerate the efficient design and modeling of antibodies and expedite the discovery of antibody-based therapeutics.

Bioinformatics
De novo protein design with a language model

Eleni Kotsiliti

DOI: https://doi.org/10.1038/s41587-022-01518-5

IF: 46.9

2022-10-08

Nature Biotechnology

Abstract:Artificial intelligence models of natural language are becoming ever more adept at processing and 'understanding' language, and are widely used in applications such as automated speech recognition, translation, smart assistants and text generation. How useful they might be for modeling biological phenomena is a question of longstanding interest among computational biologists. Recently, language models have been applied to such problems as protein function prediction, protein evolution analysis and protein design. A report in Nature Communications by Ferruz et al. advances the application to protein design, providing an unsupervised, autoregressive language model for generating de novo protein sequences.

biotechnology & applied microbiology
Active Finetuning Protein Language Model: A Budget-Friendly Method for Directed Evolution

Ming Qin,Keyan Ding,Bin Wu,Zhenping Li,Haihong Yang,Zeyuan Wang,Hongbin Ye,Haoran Yu,Huajun Chen,Qiang Zhang

2023-01-01

Abstract:Directed evolution is a widely-used strategy of protein engineering to improve protein function via mimicking natural mutation and selection. Machine learning-assisted directed evolution(MLDE) approaches aim to learn a fitness predictor, thereby efficiently searching for optimal mutants within the vast combinatorial mutation space. Since annotating mutants is both costly and labor-intensive, how to efficiently sample and utilize informative protein mutants to train the predictor is a critical problem in MLDE. Previous MLDE works just simply utilized pre-trained protein language models (PPLMs) for sampling without tailoring to the specific target protein of interest, which has not fully exploited the potential of PPLMs. In this work, we propose a novel method, the Actively-Finetuned Protein model for Directed Evolution(AFP-DE) which leverages PPLMs to actively sample and fine-tune themselves, continuously improving the model’s sampling and overall performance through iterations, to achieve efficiently directed protein evolution. Extensive experiments have shown the effectiveness of our method in generating optimal mutants with minimal annotation effort, outperforming previous works even with fewer annotated mutants, making it budget-friendly for biological experiments.
Large scale paired antibody language models

Henry Kenlay,Frédéric A. Dreyer,Aleksandr Kovaltsuk,Dom Miketa,Douglas Pires,Charlotte M. Deane

2024-03-27

Abstract:Antibodies are proteins produced by the immune system that can identify and neutralise a wide variety of antigens with high specificity and affinity, and constitute the most successful class of biotherapeutics. With the advent of next-generation sequencing, billions of antibody sequences have been collected in recent years, though their application in the design of better therapeutics has been constrained by the sheer volume and complexity of the data. To address this challenge, we present IgBert and IgT5, the best performing antibody-specific language models developed to date which can consistently handle both paired and unpaired variable region sequences as input. These models are trained comprehensively using the more than two billion unpaired sequences and two million paired sequences of light and heavy chains present in the Observed Antibody Space dataset. We show that our models outperform existing antibody and protein language models on a diverse range of design and regression tasks relevant to antibody engineering. This advancement marks a significant leap forward in leveraging machine learning, large scale data sets and high-performance computing for enhancing antibody design for therapeutic development.

Biomolecules,Machine Learning
Designing proteins with language models

Jeffrey A. Ruffolo,Ali Madani

DOI: https://doi.org/10.1038/s41587-024-02123-4

IF: 46.9

2024-02-01

Nature Biotechnology

Abstract:Protein language models learn from diverse sequences spanning the evolutionary tree and have proven to be powerful tools for sequence design, variant effect prediction and structure prediction. What are the foundations of protein language models, and how are they applied in protein engineering?

biotechnology & applied microbiology
Language models generalize beyond natural proteins

Verkuil,R.,Kabeli,O.,Du,Y.,Wicky,B. I.,Milles,L. F.,Dauparas,J.,Baker,D.,Ovchinnikov,S.,Sercu,T.,Rives,A.

DOI: https://doi.org/10.1101/2022.12.21.521521

2022-12-23

bioRxiv

Abstract:Learning the design patterns of proteins from sequences across evolution may have promise toward generative protein design. However it is unknown whether language models, trained on sequences of natural proteins, will be capable of more than memorization of existing protein families. Here we show that language models generalize beyond natural proteins to generate de novo proteins. We focus on two protein design tasks: fixed backbone design where the structure is specified, and unconstrained generation where the structure is sampled from the model. Remarkably although the models are trained only on sequences, we find that they are capable of designing structure. A total of 228 generated proteins are evaluated experimentally with high overall success rates (152/228 or 67%) in producing a soluble and monomeric species by size exclusion chromatography. Out of 152 experimentally successful designs, 35 have no significant sequence match to known natural proteins. Of the remaining 117, sequence identity to the nearest sequence match is at median 27%, below 20% for 6 designs, and as low as 18% for 3 designs. For fixed backbone design, the language model generates successful designs for each of eight experimentally evaluated artificially created fixed backbone targets. For unconstrained generation, sampled proteins cover diverse topologies and secondary structure compositions, and have high experimental success rate (71/129 or 55%). The designs reflect deep patterns linking sequence and structure, including motifs that occur in related natural structures, and motifs that are not observed in similar structural contexts in known protein families. The results show that language models, though only trained on sequences, learn a deep grammar that enables the design of protein structure, extending beyond natural proteins.
Protein language model pseudolikelihoods capture features of in vivo B cell selection and evolution

Daphne van Ginneken,Anamay Samant,Karlis Daga-Krumins,Andreas Agrafiotis,Evgenios Kladis,Sai T. Reddy,Alexander Yermanos

DOI: https://doi.org/10.1101/2024.12.09.627494

2024-12-11

Abstract:B cell selection and evolution play crucial roles in dictating successful immune responses. Recent advancements in sequencing technologies and deep-learning strategies have paved the way for generating and exploiting an ever-growing wealth of antibody repertoire data. The self-supervised nature of protein language models (PLMs) has demonstrated the ability to learn complex representations of antibody sequences and has been leveraged for a wide range of applications including diagnostics, structural modeling, and antigen-specificity predictions. PLM-derived likelihoods have been used to improve antibody affinities in vitro, raising the question of whether PLMs can capture and predict features of B cell selection in vivo. Here, we explore how general and antibody-specific PLM-generated sequence pseudolikelihoods (SPs) relate to features of in vivo B cell selection such as expansion, isotype usage, and somatic hypermutation (SHM) at single-cell resolution. Our results demonstrate that the type of PLM and the region of the antibody input sequence significantly affect the generated SP. Contrary to previous in vitro reports, we observe a negative correlation between SPs and binding affinity, whereas repertoire features such as SHM, isotype usage, and antigen specificity were strongly correlated with SPs. By constructing evolutionary lineage trees of B cell clones from human and mouse repertoires, we observe that SHMs are routinely among the most likely mutations suggested by PLMs and that mutating residues have lower absolute likelihoods than conserved residues. Our findings highlight the potential of PLMs to predict features of antibody selection and further suggest their potential to assist in antibody discovery and engineering.

Bioinformatics
Reprogramming Pretrained Language Models for Antibody Sequence Infilling

Igor Melnyk,Vijil Chenthamarakshan,Pin-Yu Chen,Payel Das,Amit Dhurandhar,Inkit Padhi,Devleena Das

2023-06-20

Abstract:Antibodies comprise the most versatile class of binding molecules, with numerous applications in biomedicine. Computational design of antibodies involves generating novel and diverse sequences, while maintaining structural consistency. Unique to antibodies, designing the complementarity-determining region (CDR), which determines the antigen binding affinity and specificity, creates its own unique challenges. Recent deep learning models have shown impressive results, however the limited number of known antibody sequence/structure pairs frequently leads to degraded performance, particularly lacking diversity in the generated sequences. In our work we address this challenge by leveraging Model Reprogramming (MR), which repurposes pretrained models on a source language to adapt to the tasks that are in a different language and have scarce data - where it may be difficult to train a high-performing model from scratch or effectively fine-tune an existing pre-trained model on the specific task. Specifically, we introduce ReprogBert in which a pretrained English language model is repurposed for protein sequence infilling - thus considers cross-language adaptation using less data. Results on antibody design benchmarks show that our model on low-resourced antibody sequence dataset provides highly diverse CDR sequences, up to more than a two-fold increase of diversity over the baselines, without losing structural integrity and naturalness. The generated sequences also demonstrate enhanced antigen binding specificity and virus neutralization ability. Code is available at <a class="link-external link-https" href="https://github.com/IBM/ReprogBERT" rel="external noopener nofollow">this https URL</a>

Biomolecules,Machine Learning
Protein Design by Directed Evolution Guided by Large Language Models

Trong Thanh Tran,Truong Son Hy

DOI: https://doi.org/10.1101/2023.11.28.568945

2024-08-18

Abstract:Directed evolution, a strategy for protein engineering, optimizes protein properties (i.e., fitness) by a rigorous and resource-intensive process of screening or selecting among a vast range of mutations. By conducting an in silico screening of sequence properties, machine learning-guided directed evolution (MLDE) can expedite the optimization process and alleviate the experimental workload. In this work, we propose a general MLDE framework in which we apply recent advancements of Deep Learning in protein representation learning and protein property prediction to accelerate the searching and optimization processes. In particular, we introduce an optimization pipeline that utilizes Large Language Models (LLMs) to pinpoint the mutation hotspots in the sequence and then suggest replacements to improve the overall fitness. Our experiments have shown the superior efficiency and efficacy of our proposed framework in the conditional protein generation, in comparision with other state-of-the-art baseline algorithms. We expect this work will shed a new light on not only protein engineering but also on solving combinatorial problems using data-driven methods. Our implementation is publicly available at https://github.com/HySonLab/Directed_Evolution

Bioinformatics
Protein language models enable prediction of polyreactivity of monospecific, bispecific, and heavy-chain-only antibodies

Xin Yu,Kostika Vangjeli,Anusha Prakash,Meha Chhaya,Samantha J Stanley,Noah Cohen,Lili Huang,Yu,X.,Vangjeli,K.,Prakash,A.,Chhaya,M.,Stanley,S.,Cohen,N.,Huang,L.

DOI: https://doi.org/10.1101/2023.11.06.565888

2023-11-08

bioRxiv

Abstract:Early assessment of antibody off-target binding is essential for mitigating developability risks such as fast clearance, reduced efficacy, toxicity, and immunogenicity. The baculovirus particle (BVP) binding assay has been widely utilized to evaluate polyreactivity of antibodies. As a complementary approach, computational prediction of polyreactivity is desirable for counter-screening antibodies from in silico discovery campaigns. However, there is a lack of such models. Herein, we present the development of an ensemble of three deep learning models based on two pan-protein foundational protein language models (ESM2 and ProtT5) and an antibody-specific protein language model (Antiberty). These models were trained in a transfer learning network to predict the outcomes in the BVP assay and the bovine serum albumin (BSA) binding assay which was developed as a complement to the BVP assay. The training was conducted on a large dataset of antibody sequences augmented with experimental conditions, which were collected through a highly efficient application system. The resulting models demonstrated robust performance on normal mAbs (monospecific with heavy and light chain), bispecific Abs, and single-domain Fc (VHH-Fc). Protein language models outperformed a model built using molecular descriptors calculated from AlphaFold 2 predicted structures. Embeddings from the antibody-specific and foundational protein language models resulted in similar performance. To our knowledge, this represents the first application of protein language models to predict assay data on bispecifics and VHH-Fcs. Our study yields valuable insights on building infrastructures to support machine learning activities and training models for critical assays in antibody discovery.

English Else
InstructPLM: Aligning Protein Language Models to Follow Protein Structure Instructions

Jiezhong Qiu,Junde Xu,Jie Hu,Hanqun Cao,Liya Hou,Zijun Gao,Xinyi Zhou,Anni Li,Xiujuan Li,Bin Cui,Fei Yang,Shuang Peng,Ning Sun,Fangyu Wang,Aimin Pan,Jie Tang,Jieping Ye,Junyang Lin,Jin Tang,Xingxu Huang,Pheng Ann Heng,Guangyong Chen

DOI: https://doi.org/10.1101/2024.04.17.589642

2024-04-20

Abstract:Large language models are renowned for their efficacy in capturing intricate patterns, including co-evolutionary relationships, and underlying protein languages. However, current methodologies often fall short in illustrating the emergence of genomic insertions, duplications, and insertion/deletions (indels), which account for approximately 14% of human pathogenic mutations. Given that structure dictates function, mutated proteins with similar structures are more likely to persist throughout biological evolution. Motivated by this, we leverage crossmodality alignment and instruct fine-tuning techniques inspired by large language models to align a generative protein language model with protein structure instructions. Specifically, we present a method for generating variable-length and diverse proteins to explore and simulate the complex evolution of life, thereby expanding the repertoire of options for protein engineering. Our proposed protein LM-based approach, InstructPLM, demonstrates significant performance enhancements both in silico and in vitro. On native protein backbones, it achieves a perplexity of 2.68 and a sequence recovery rate of 57.51, surpassing Protein-MPNN by 39.2% and 25.1%, respectively. Furthermore, we validate the efficacy of our model by redesigning PETase and L-MDH. For PETase, all fifteen designed variable-length PETase exhibit depolymerization activity, with eleven surpassing the activity levels of the wild type. Regarding L-MDH, an enzyme lacking an experimentally determined structure, InstructPLM is able to design functional enzymes with an AF2-predicted structure. Code and model weights of InstructPLM are publicly available .

Bioinformatics
Simulating 500 million years of evolution with a language model

Tomas Hayes,Roshan Rao,Halil Akin,Nicholas J Sofroniew,Deniz Oktay,Zeming Lin,Robert Verkuil,Vincent Q Tran,Jonathan Deaton,Marius Wiggert,Rohil Badkundri,Irhum Shafkat,Jun Gong,Alexander Derry,Raúl Santiago Molina,Neil Thomas,Yousuf A Khan,Chetan Mishra,Carolyn Kim,Liam J Bartie,Matthew Nemeth,Patrick D Hsu,Tom Sercu,Salvatore Candido,Alexander Rives

DOI: https://doi.org/10.1101/2024.07.01.600583

2024-07-02

Abstract:More than three billion years of evolution have produced an image of biology encoded into the space of natural proteins. Here we show that language models trained on tokens generated by evolution can act as evolutionary simulators to generate functional proteins that are far away from known proteins. We present ESM3, a frontier multimodal generative language model that reasons over the sequence, structure, and function of proteins. ESM3 can follow complex prompts combining its modalities and is highly responsive to biological alignment. We have prompted ESM3 to generate fluorescent proteins with a chain of thought. Among the generations that we synthesized, we found a bright fluorescent protein at far distance (58% identity) from known fluorescent proteins. Similarly distant natural fluorescent proteins are separated by over five hundred million years of evolution.

Synthetic Biology
On Pre-trained Language Models for Antibody

Danqing Wang,Fei Ye,Hao Zhou

DOI: https://doi.org/10.48550/arXiv.2301.12112

2023-01-28

Computation and Language

Abstract:Antibodies are vital proteins offering robust protection for the human body from pathogens. The development of general protein and antibody-specific pre-trained language models both facilitate antibody prediction tasks. However, there have been limited studies that comprehensively explore the representation capability of distinct pre-trained language models on different antibody tasks. To investigate the problem, we aim to answer several key questions in this paper, such as how pre-trained language models perform in antibody tasks with different specificity and how introducing specific biological mechanisms to the pre-training process can benefit the model. Additionally, we evaluate if the learned antibody pre-trained representations can be applied to real-world antibody problems, like drug discovery and immune process understanding. Previously, no benchmark available largely hindered the study to answer these questions. To aid in our investigation, we provide an AnTibody Understanding Evaluation (ATUE) benchmark. We comprehensively evaluate the performance of protein pre-trained language models by empirical study along with conclusions and new insights. Our ATUE and code are released at https://github.com/dqwang122/EATLM.
From a single sequence to evolutionary trajectories: protein language models capture the evolutionary potential of SARS-CoV-2 protein sequences

Kieran D. Lamb,Joseph Hughes,Spyros Lytras,Francesca Young,Orges Koci,James Herzig,Simon C Lovell,Joe Grove,Ke Yuan,David L. Robertson

DOI: https://doi.org/10.1101/2024.07.05.602129

2024-09-18

Abstract:Protein language models (PLMs) capture features of protein three-dimensional structure from amino acid sequences alone, without requiring multiple sequence alignments (MSA). The concepts of grammar and semantics from natural language have the potential to capture functional properties of proteins. Here, we investigate how these representations enable assessment of variation due to mutation. Applied to SARS-CoV-2’s spike protein using in silico deep mutational scanning (DMS) we demonstrate the PLM, ESM-2, has learned the sequence context within which variation occurs, capturing evolutionary constraint. This recapitulates what conventionally requires MSA data to predict. Unlike other state-of-the-art methods which require protein structures or multiple sequences for training, we show what can be accomplished using an unmodified pretrained PLM. We demonstrate that the grammaticality and semantic scores represent novel metrics. Applied to SARS-CoV-2 variants across the pandemic we show that ESM-2 representations encode the evolutionary history between variants, as well as the distinct nature of variants of concern upon their emergence, associated with shifts in receptor binding and antigenicity. PLM likelihoods can also identify epistatic interactions among sites in the protein. Our results here affirm that PLMs are broadly useful for variant-effect prediction, including unobserved changes, and can be applied to understand novel viral pathogens with the potential to be applied to any protein sequence, pathogen or otherwise.

Bioinformatics

Protein language models guide directed antibody evolution

Protein Language Models in Directed Evolution

Addressing the antibody germline bias and its effect on language models for improved antibody design

Learning the protein language: Evolution, structure, and function

Knowledge-aware Reinforced Language Models for Protein Directed Evolution

Learning the Language of Antibody Hypervariability

De novo protein design with a language model

Active Finetuning Protein Language Model: A Budget-Friendly Method for Directed Evolution

Large scale paired antibody language models

Designing proteins with language models

Language models generalize beyond natural proteins

Protein language model pseudolikelihoods capture features of in vivo B cell selection and evolution

Reprogramming Pretrained Language Models for Antibody Sequence Infilling

Protein Design by Directed Evolution Guided by Large Language Models

Protein language models enable prediction of polyreactivity of monospecific, bispecific, and heavy-chain-only antibodies

InstructPLM: Aligning Protein Language Models to Follow Protein Structure Instructions

Simulating 500 million years of evolution with a language model

On Pre-trained Language Models for Antibody

From a single sequence to evolutionary trajectories: protein language models capture the evolutionary potential of SARS-CoV-2 protein sequences