Protein sequence design by deep learning

Jue Wang

DOI: https://doi.org/10.1038/s43588-022-00274-5

2022-07-22

Nature Computational Science

Abstract:The design of protein sequences that can precisely fold into pre-specified 3D structures is a challenging task. A recently proposed deep-learning algorithm improves such designs when compared with traditional, physics-based protein design approaches.

What problem does this paper attempt to address?

Protein sequence design on given backbones with deep learning

Yufeng Liu,Haiyan Liu

DOI: https://doi.org/10.1093/protein/gzad024

2024-01-29

Abstract:Deep learning methods for protein sequence design focus on modeling and sampling the many- dimensional distribution of amino acid sequences conditioned on the backbone structure. To produce physically foldable sequences, inter-residue couplings need to be considered properly. These couplings are treated explicitly in iterative methods or autoregressive methods. Non-autoregressive models treating these couplings implicitly are computationally more efficient, but still await tests by wet experiment. Currently, sequence design methods are evaluated mainly using native sequence recovery rate and native sequence perplexity. These metrics can be complemented by sequence-structure compatibility metrics obtained from energy calculation or structure prediction. However, existing computational metrics have important limitations that may render the generalization of computational test results to performance in real applications unwarranted. Validation of design methods by wet experiments should be encouraged.
Protein design via deep learning

Wenze Ding,Kenta Nakai,Haipeng Gong

DOI: https://doi.org/10.1093/bib/bbac102

IF: 9.5

2022-03-25

Briefings in Bioinformatics

Abstract:Abstract Proteins with desired functions and properties are important in fields like nanotechnology and biomedicine. De novo protein design enables the production of previously unseen proteins from the ground up and is believed as a key point for handling real social challenges. Recent introduction of deep learning into design methods exhibits a transformative influence and is expected to represent a promising and exciting future direction. In this review, we retrospect the major aspects of current advances in deep-learning-based design procedures and illustrate their novelty in comparison with conventional knowledge-based approaches through noticeable cases. We not only describe deep learning developments in structure-based protein design and direct sequence design, but also highlight recent applications of deep reinforcement learning in protein design. The future perspectives on design goals, challenges and opportunities are also comprehensively discussed.

biochemical research methods,mathematical & computational biology
Computational Protein Design with Deep Learning Neural Networks

Jingxue Wang,Huali Cao,John Z. H. Zhang,Yifei Qi

DOI: https://doi.org/10.1038/s41598-018-24760-x

IF: 4.6

2018-04-20

Scientific Reports

Abstract:Computational protein design has a wide variety of applications. Despite its remarkable success, designing a protein for a given structure and function is still a challenging task. On the other hand, the number of solved protein structures is rapidly increasing while the number of unique protein folds has reached a steady number, suggesting more structural information is being accumulated on each fold. Deep learning neural network is a powerful method to learn such big data set and has shown superior performance in many machine learning fields. In this study, we applied the deep learning neural network approach to computational protein design for predicting the probability of 20 natural amino acids on each residue in a protein. A large set of protein structures was collected and a multi-layer neural network was constructed. A number of structural properties were extracted as input features and the best network achieved an accuracy of 38.3%. Using the network output as residue type restraints improves the average sequence identity in designing three natural proteins using Rosetta. Moreover, the predictions from our network show ~3% higher sequence identity than a previous method. Results from this study may benefit further development of computational protein design methods.

multidisciplinary sciences
Structure-based protein design with deep learning

Sergey Ovchinnikov,Po-Ssu Huang

DOI: https://doi.org/10.1016/j.cbpa.2021.08.004

IF: 8.972

2021-12-01

Current Opinion in Chemical Biology

Abstract:Since the first revelation of proteins functioning as macromolecular machines through their three dimensional structures, researchers have been intrigued by the marvelous ways the biochemical processes are carried out by proteins. The aspiration to understand protein structures has fueled extensive efforts across different scientific disciplines. In recent years, it has been demonstrated that proteins with new functionality or shapes can be designed via structure-based modeling methods, and the design strategies have combined all available information — but largely piece-by-piece — from sequence derived statistics to the detailed atomic-level modeling of chemical interactions. Despite the significant progress, incorporating data-derived approaches through the use of deep learning methods can be a game changer. In this review, we summarize current progress, compare the arc of developing the deep learning approaches with the conventional methods, and describe the motivation and concepts behind current strategies that may lead to potential future opportunities.

biochemistry & molecular biology,biophysics
All-atom protein sequence design based on geometric deep learning

Jiale Liu,Zheng Guo,Changsheng Zhang,Luhua Lai

DOI: https://doi.org/10.1101/2024.03.18.585651

2024-03-19

Abstract:The development of advanced deep learning methods has revolutionized computational protein design. Although the success rate of design has been significantly increased, the overall accuracy of design remains low. Many computational sequence design approaches are devoted to recover the original sequences for given protein structures by encoding the environment of the central residue without considering atomic details of side chains. This may limit the exploration of new sequences that can fold into the same structure and restrain function design that depends on interaction details. In this study, we proposed a novel deep learning frame-work, GeoSeqBuilder, to learn the relationship between protein structure and sequence based on rotational and translational invariance by extracting the information from relative locations. We utilized geometric deep learning to fetch the spatial local geometric features from protein backbones and explicitly incorporated three-body interactions to learn the inter-residue coupling information, and then determined the central residue type. Our model recovers over 50% native residue types and simultaneously gives highly accurate prediction of side-chain conformations which gives the atomic interaction details and circumvents the dependence of protein structure prediction tools. We used the likelihood confidence log as scoring function for sequence and structure consistence evaluation which exhibits strong correlation with TM-score, and can be applied to recognize near-native structures from protein decoys pool in protein structure prediction. We have used GeoSeqBuilder to design sequences for two proteins, including thiore-doxin and a hallucinated protein. All of the 15 sequences experimentally tested can be expressed as soluble monomeric proteins with high thermal stability and correct secondary structures. We further solved one crystal structure for thioredoxin and two for the hallucinated structure and all the experimentally solved structures are in good agreement with the designed models. The two designed sequences for the hallucination structure are novel without any homologous sequences within the latest released database clust30. The ability of GeoSeqBuilder to design new sequences for given protein structures with atomic details makes it applicable, not only for sequence design, but also for protein-protein interaction and functional protein design.

Biophysics
From sequence to function through structure: deep learning for protein design

Noelia Ferruz,Michael Heinzinger,Mehmet Akdel,Alexander Goncearenco,Luca Naef,Christian Dallago

DOI: https://doi.org/10.1016/j.csbj.2022.11.014

IF: 6.155

2022-11-20

Computational and Structural Biotechnology Journal

Abstract:The process of designing biomolecules, in particular proteins, is witnessing a rapid change in available tooling and approaches, moving from design through physicochemical force fields, to producing plausible, complex sequences fast via end-to-end differentiable statistical models. To achieve conditional and controllable protein design, researchers at the interface of artificial intelligence and biology leverage advances in natural language processing (NLP) and computer vision techniques, coupled with advances in computing hardware to learn patterns from growing biological databases, curated annotations thereof, or both. Once learned, these patterns can be leveraged to provide novel insights into mechanistic biology and the design of biomolecules. However, navigating and understanding the practical applications for the many recent protein design tools is complex. To facilitate this, we 1) document recent advances in deep learning (DL) assisted protein design from the last three years, 2) present a practical pipeline that allows to go from de novo -generated sequences to their predicted properties and web-powered visualization within minutes, and 3) leverage it to suggest a generated protein sequence which might be used to engineer a biosynthetic gene cluster to produce a molecular glue-like compound. Lastly, we discuss challenges and highlight opportunities for the protein design field.

biochemistry & molecular biology
Protein sequence design with a learned potential

Namrata Anand,Raphael Eguchi,Irimpan I. Mathews,Carla P. Perez,Alexander Derry,Russ B. Altman,Po-Ssu Huang

DOI: https://doi.org/10.1038/s41467-022-28313-9

IF: 16.6

2022-02-08

Nature Communications

Abstract:Abstract The task of protein sequence design is central to nearly all rational protein engineering problems, and enormous effort has gone into the development of energy functions to guide design. Here, we investigate the capability of a deep neural network model to automate design of sequences onto protein backbones, having learned directly from crystal structure data and without any human-specified priors. The model generalizes to native topologies not seen during training, producing experimentally stable designs. We evaluate the generalizability of our method to a de novo TIM-barrel scaffold. The model produces novel sequences, and high-resolution crystal structures of two designs show excellent agreement with in silico models. Our findings demonstrate the tractability of an entirely learned method for protein sequence design.

multidisciplinary sciences
ZetaDesign: an end-to-end deep learning method for protein sequence design and side-chain packing

Junyu Yan,Shuai Li,Ying Zhang,Aimin Hao,Qinping Zhao

DOI: https://doi.org/10.1093/bib/bbad257

IF: 9.5

2023-07-11

Briefings in Bioinformatics

Abstract:Computational protein design has been demonstrated to be the most powerful tool in the last few years among protein designing and repacking tasks. In practice, these two tasks are strongly related but often treated separately. Besides, state-of-the-art deep-learning-based methods cannot provide interpretability from an energy perspective, affecting the accuracy of the design. Here we propose a new systematic approach, including both a posterior probability and a joint probability parts, to solve the two essential questions once for all. This approach takes the physicochemical property of amino acids into consideration and uses the joint probability model to ensure the convergence between structure and amino acid type. Our results demonstrated that this method could generate feasible, high-confidence sequences with low-energy side conformations. The designed sequences can fold into target structures with high confidence and maintain relatively stable biochemical properties. The side chain conformation has a significantly lower energy landscape without delegating to a rotamer library or performing the expensive conformational searches. Overall, we propose an end-to-end method that combines the advantages of both deep learning and energy-based methods. The design results of this model demonstrate high efficiency, and precision, as well as a low energy state and good interpretability.

biochemical research methods,mathematical & computational biology
A new age in protein design empowered by deep learning

Hamed Khakzad,Ilia Igashov,Arne Schneuing,Casper Goverde,Michael Bronstein,Bruno Correia

DOI: https://doi.org/10.1016/j.cels.2023.10.006

IF: 11.091

2023-11-01

Cell Systems

Abstract:The rapid progress in the field of deep learning has had a significant impact on protein design. Deep learning methods have recently produced a breakthrough in protein structure prediction, leading to the availability of high-quality models for millions of proteins. Along with novel architectures for generative modeling and sequence analysis, they have revolutionized the protein design field in the past few years remarkably by improving the accuracy and ability to identify novel protein sequences and structures. Deep neural networks can now learn and extract the fundamental features of protein structures, predict how they interact with other biomolecules, and have the potential to create new effective drugs for treating disease. As their applicability in protein design is rapidly growing, we review the recent developments and technology in deep learning methods and provide examples of their performance to generate novel functional proteins.

cell biology,biochemistry & molecular biology
Robust deep learning based protein sequence design using ProteinMPNN

D. Tischer,A. Courbet,T. Huddy,B. Sankaran,D. Baker,L. Milles,S. Pellock,A. Bera,N. Bennett,P. J. Leung,J. Dauparas,F. Chan,B. Koepnick,R. Ragotte,I. Anishchenko,H. Bai,A. Kang,N. King,H. Nguyen,B. Wicky,R. D. de Haas,N. Bethel

DOI: https://doi.org/10.1126/science.add2187

2022-06-04

bioRxiv

Abstract:While deep learning has revolutionized protein structure prediction, almost all experimentally characterized de novo protein designs have been generated using physically based approaches such as Rosetta. Here we describe a deep learning based protein sequence design method, ProteinMPNN, with outstanding performance in both in silico and experimental tests. The amino acid sequence at different positions can be coupled between single or multiple chains, enabling application to a wide range of current protein design challenges. On native protein backbones, ProteinMPNN has a sequence recovery of 52.4%, compared to 32.9% for Rosetta. Incorporation of noise during training improves sequence recovery on protein structure models, and produces sequences which more robustly encode their structures as assessed using structure prediction algorithms. We demonstrate the broad utility and high accuracy of ProteinMPNN using X-ray crystallography, cryoEM and functional studies by rescuing previously failed designs, made using Rosetta or AlphaFold, of protein monomers, cyclic homo-oligomers, tetrahedral nanoparticles, and target binding proteins. One-sentence summary A deep learning based protein sequence design method is described that is widely applicable to current design challenges and shows outstanding performance in both in silico and experimental tests.

Computer Science,Medicine,Biology
Deep learning for protein structure prediction and design—progress and applications

Jürgen Jänes,Pedro Beltrao

DOI: https://doi.org/10.1038/s44320-024-00016-x

IF: 13.068

2024-01-30

Molecular Systems Biology

Abstract:Abstract Proteins are the key molecular machines that orchestrate all biological processes of the cell. Most proteins fold into three-dimensional shapes that are critical for their function. Studying the 3D shape of proteins can inform us of the mechanisms that underlie biological processes in living cells and can have practical applications in the study of disease mutations or the discovery of novel drug treatments. Here, we review the progress made in sequence-based prediction of protein structures with a focus on applications that go beyond the prediction of single monomer structures. This includes the application of deep learning methods for the prediction of structures of protein complexes, different conformations, the evolution of protein structures and the application of these methods to protein design. These developments create new opportunities for research that will have impact across many areas of biomedical research.

biochemistry & molecular biology
All‐Atom Protein Sequence Design Based on Geometric Deep Learning

Jiale Liu,Zheng Guo,Hantian You,Changsheng Zhang,Luhua Lai

DOI: https://doi.org/10.1002/anie.202411461

2024-09-20

Angewandte Chemie International Edition

Abstract:Designing sequences for specific protein backbones is a key step in creating new functional proteins. Here, we introduce GeoSeqBuilder, a deep learning framework that integrates protein sequence generation with side chain conformation prediction to produce the complete all-atom structures for designed sequences. GeoSeqBuilder uses spatial geometric features from protein backbones and explicitly includes three-body interactions of neighboring residues. GeoSeqBuilder achieves native residue type recovery rate of 51.6%, comparable to ProteinMPNN and other leading methods, while accurately predicting side chain conformations. We first used GeoSeqBuilder to design sequences for thioredoxin and a hallucinated three-helical bundle protein. All the 15 tested sequences expressed as soluble monomeric proteins with high thermal stability, and the 2 high-resolution crystal structures solved closely match the designed models. The generated protein sequences exhibit low similarity (minimum 23%) to the original sequences, with significantly altered hydrophobic cores. We further redesigned the hydrophobic core of glutathione peroxidase 4, and 3 of the 5 designs showed improved enzyme activity. Although further testing is needed, the high experimental success rate in our testing demonstrates that GeoSeqBuilder is a powerful tool for designing novel sequences for predefined protein structures with atomic details. GeoSeqBuilder is available at https://github.com/PKUliujl/GeoSeqBuilder

chemistry, multidisciplinary
Protein sequence design with deep generative models

Zachary Wu,Kadina E. Johnston,Frances H. Arnold,Kevin K. Yang

DOI: https://doi.org/10.1016/j.cbpa.2021.04.004

2021-04-10

Abstract:Protein engineering seeks to identify protein sequences with optimized properties. When guided by machine learning, protein sequence generation methods can draw on prior knowledge and experimental efforts to improve this process. In this review, we highlight recent applications of machine learning to generate protein sequences, focusing on the emerging field of deep generative methods.

Quantitative Methods,Machine Learning,Biomolecules
Deep Learning in Protein Structural Modeling and Design

Wenhao Gao,Sai Pooja Mahajan,Jeremias Sulam,Jeffrey J. Gray

DOI: https://doi.org/10.48550/arXiv.2007.08383

2020-07-16

Abstract:Deep learning is catalyzing a scientific revolution fueled by big data, accessible toolkits, and powerful computational resources, impacting many fields including protein structural modeling. Protein structural modeling, such as predicting structure from amino acid sequence and evolutionary information, designing proteins toward desirable functionality, or predicting properties or behavior of a protein, is critical to understand and engineer biological systems at the molecular level. In this review, we summarize the recent advances in applying deep learning techniques to tackle problems in protein structural modeling and design. We dissect the emerging approaches using deep learning techniques for protein structural modeling, and discuss advances and challenges that must be addressed. We argue for the central importance of structure, following the "sequence -> structure -> function" paradigm. This review is directed to help both computational biologists to gain familiarity with the deep learning methods applied in protein modeling, and computer scientists to gain perspective on the biologically meaningful problems that may benefit from deep learning techniques.

Biomolecules,Machine Learning
Highly accurate and robust protein sequence design with CarbonDesign

D. Bu,Haicang Zhang,Chungong Yu,Milong Ren

DOI: https://doi.org/10.1101/2023.08.07.552204

2023-08-07

bioRxiv

Abstract:Protein sequence design, the inverse problem of protein structure prediction, plays a crucial role in protein engineering. Although recent deep learning-based methods have shown promising advancements, achieving accurate and robust protein sequence design remains an ongoing challenge. Here, we present CarbonDesign, a new approach that draws inspiration from successful ingredients of AlphaFold for protein structure prediction and makes significant and novel developments tailored specifically for protein sequence design. At its core, CarbonDesign explores Inverseformer, a novel network architecture adapted from AlphaFold’s Evoformer, to learn representations from backbone structures and an amortized Markov Random Fields model for sequence decoding. Moreover, we incorporate other essential AlphaFold concepts into CarbonDesign: an end-to-end network recycling technique to leverage evolutionary constraints in protein language models and a multi-task learning technique to generate side chain structures corresponding to the designed sequences. Through rigorous evaluations on independent testing data sets, including the CAMEO and recent CASP15 data sets, as well as the predicted structures from AlphaFold, we show that CarbonDesign outperforms other published methods, achieving high accuracy in sequence generation. Moreover, it exhibits superior performance on de novo backbone structures obtained from recent diffusion generative models such as RFdiffusion and FrameDiff, highlighting its potential for enhancing de novo protein design. Notably, CarbonDesign also supports zero-shot prediction of the functional effects of sequence variants, indicating its potential application in directed evolution-based design. In summary, our results illustrate CarbonDesign’s accurate and robust performance in protein sequence design, making it a promising tool for applications in bioengineering.

Biology,Computer Science
Deep learning techniques have significantly impacted protein structure prediction and protein design

Robin Pearce,Yang Zhang

DOI: https://doi.org/10.1016/j.sbi.2021.01.007

IF: 7.786

2021-06-01

Current Opinion in Structural Biology

Abstract:<p>Protein structure prediction and design can be regarded as two inverse processes governed by the same folding principle. Although progress remained stagnant over the past two decades, the recent application of deep neural networks to spatial constraint prediction and end-to-end model training has significantly improved the accuracy of protein structure prediction, largely solving the problem at the fold level for single-domain proteins. The field of protein design has also witnessed dramatic improvement, where noticeable examples have shown that information stored in neural-network models can be used to advance functional protein design. Thus, incorporation of deep learning techniques into different steps of protein folding and design approaches represents an exciting future direction and should continue to have a transformative impact on both fields.</p>

cell biology,biochemistry & molecular biology
Accurate and robust protein sequence design with CarbonDesign

Milong Ren,Chungong Yu,Dongbo Bu,Haicang Zhang

DOI: https://doi.org/10.1038/s42256-024-00838-2

IF: 23.8

2024-05-24

Nature Machine Intelligence

Abstract:Protein sequence design is critically important for protein engineering. Despite recent advancements in deep learning-based methods, achieving accurate and robust sequence design remains a challenge. Here we present CarbonDesign, an approach that draws inspiration from successful ingredients of AlphaFold and which has been developed specifically for protein sequence design. At its core, CarbonDesign introduces Inverseformer, which learns representations from backbone structures and an amortized Markov random fields model for sequence decoding. Moreover, we incorporate other essential AlphaFold concepts into CarbonDesign: an end-to-end network recycling technique to leverage evolutionary constraints from protein language models and a multitask learning technique for generating side-chain structures alongside designed sequences. CarbonDesign outperforms other methods on independent test sets including the 15th Critical Assessment of protein Structure Prediction (CASP15) dataset, the Continuous Automated Model Evaluation (CAMEO) dataset and de novo proteins from RFDiffusion. Furthermore, it supports zero-shot prediction of the functional effects of sequence variants, making it a promising tool for applications in bioengineering.

computer science, artificial intelligence, interdisciplinary applications
Protein Design Using Physics Informed Neural Networks

Omar,Keasar,Ben-Sasson,Haber

DOI: https://doi.org/10.3390/biom13030457

IF: 6.064

2023-03-02

Biomolecules

Abstract:The inverse protein folding problem, also known as protein sequence design, seeks to predict an amino acid sequence that folds into a specific structure and performs a specific function. Recent advancements in machine learning techniques have been successful in generating functional sequences, outperforming previous energy function-based methods. However, these machine learning methods are limited in their interoperability and robustness, especially when designing proteins that must function under non-ambient conditions, such as high temperature, extreme pH, or in various ionic solvents. To address this issue, we propose a new Physics-Informed Neural Networks (PINNs)-based protein sequence design approach. Our approach combines all-atom molecular dynamics simulations, a PINNs MD surrogate model, and a relaxation of binary programming to solve the protein design task while optimizing both energy and the structural stability of proteins. We demonstrate the effectiveness of our design framework in designing proteins that can function under \mbox{non-ambient conditions.

biochemistry & molecular biology
Protein Design Using Structure-Prediction Networks: AlphaFold and RoseTTAFold as Protein Structure Foundation Models

Jue Wang,Joseph L Watson,Sidney L Lisanza,Joseph L. Watson,Sidney L. Lisanza

DOI: https://doi.org/10.1101/cshperspect.a041472

IF: 7.2

2024-03-04

Cold Spring Harbor Perspectives in Biology

Abstract:Designing proteins with tailored structures and functions is a long-standing goal in bioengineering. Recently, deep learning advances have enabled protein structure prediction at near-experimental accuracy, which has catalyzed progress in protein design as well. We review recent studies that use structure-prediction neural networks to design proteins, via approaches such as activation maximization, inpainting, or denoising diffusion. These methods have led to major improvements over previous methods in wet-lab success rates for designing protein binders, metalloproteins, enzymes, and oligomeric assemblies. These results show that structure-prediction models are a powerful foundation for developing protein-design tools and suggest that continued improvement of their accuracy and generality will be key to unlocking the full potential of protein design.

cell biology
Designing novel protein structures using sequence generator and AlphaFold2

Xeerak Agha,Nihang Fu,Jianjun Hu

DOI: https://doi.org/10.48550/arXiv.2208.14526

2022-08-31

Abstract:Protein structures and functions are determined by a contiguous arrangement of amino acid sequences. Designing novel protein sequences and structures with desired geometry and functions is a complex task with large state spaces. Here we develop a novel protein design pipeline consisting of two deep learning algorithms, ProteinSolver and AlphaFold2. ProteinSolver is a deep graph neural network that generates amino acid sequences such that the forces between interacting amino acids are favorable and compatible with the fold while AlphaFold2 is a deep learning algorithm that predicts the protein structures from protein sequences. We present forty de novo designed binding sites of the PTP1B and P53 proteins with high precision, out of which thirty proteins are novel. Using ProteinSolver and AlphaFold2 in conjunction, we can trim the exploration of the large protein conformation space, thus expanding the ability to find novel and diverse de novo protein designs.

Chemical Physics,Biomolecules

Protein sequence design by deep learning

Protein sequence design on given backbones with deep learning

Protein design via deep learning

Computational Protein Design with Deep Learning Neural Networks

Structure-based protein design with deep learning

All-atom protein sequence design based on geometric deep learning

From sequence to function through structure: deep learning for protein design

Protein sequence design with a learned potential

ZetaDesign: an end-to-end deep learning method for protein sequence design and side-chain packing

A new age in protein design empowered by deep learning

Robust deep learning based protein sequence design using ProteinMPNN

Deep learning for protein structure prediction and design—progress and applications

All‐Atom Protein Sequence Design Based on Geometric Deep Learning

Protein sequence design with deep generative models

Deep Learning in Protein Structural Modeling and Design

Highly accurate and robust protein sequence design with CarbonDesign

Deep learning techniques have significantly impacted protein structure prediction and protein design

Accurate and robust protein sequence design with CarbonDesign

Protein Design Using Physics Informed Neural Networks

Protein Design Using Structure-Prediction Networks: AlphaFold and RoseTTAFold as Protein Structure Foundation Models

Designing novel protein structures using sequence generator and AlphaFold2