Abstract:De novo drug design refers to the process of designing new drug molecules from scratch using computational methods. In contrast to other computational methods that primarily focus on modifying existing molecules, designing from scratch enables the exploration of new chemical space and the potential discovery of novel molecules with enhanced properties. In this research, we proposed a model that utilizes Generative Pre-trained Transformer (GPT) architecture and relative attention for de novo drug design. GPT is a language model that utilizes transformer architecture to predict the next word or token in a given sequence. Representation of molecules using SMILES notation has enabled the use of next-token prediction techniques in de novo drug design. GPT uses attention mechanisms to capture the dependencies and relationships between different tokens in a sequence and allows the model to focus on the most important information when processing the input. Relative attention is a variant of the attention mechanism, which allows the model to capture the relative distances and relationships between tokens in the input sequence. In the standard attention mechanism, positional information is typically encoded using fixed-position embeddings. In relative attention, positional information is supplied dynamically during attention calculation by incorporating relative positional encodings, enabling the model to quickly learn the syntax of new unseen tokens. Relative attention enables the GPT model to better understand the relative positions of tokens in the sequence, which can be particularly useful when dealing with limited dataset sizes or generating target-specific drugs. The proposed model was trained on benchmark datasets, and performance was compared with other generative models. We show that relative attention and transfer learning could enable the GPT model to generate molecules with improved validity, uniqueness, and novelty in the context of de novo drug design. To illustrate the effectiveness of relative attention, the model was trained using transfer learning on three target-specific datasets, and the performance was compared with standard attention.

Transformer neural network for protein-specific de novo drug generation as a machine translation problem

A Transformer-based Generative Model for De Novo Molecular Design

AlphaDrug: Protein Target Specific De Novo Molecular Generation

De Novo Drug Design Using Transformer-Based Machine Translation and Reinforcement Learning of an Adaptive Monte Carlo Tree Search

Seq2Mol: Automatic design of de novo molecules conditioned by the target protein sequences through deep neural networks

De Novo Drug Design with Joint Transformers

Transformer-Based Generative Model Accelerating the Development of Novel BRAF Inhibitors

Deep generative model for drug design from protein target sequence

cMolGPT: A Conditional Generative Pre-Trained Transformer for Target-Specific De Novo Molecular Generation

Generative Pretrained Autoregressive Transformer Graph Neural Network applied to the Analysis and Discovery of Novel Proteins

Tailoring Molecules for Protein Pockets: a Transformer-based Generative Solution for Structured-based Drug Design

TD-GPT: Target Protein-Specific Drug Molecule Generation GPT

DNMG: Deep Molecular Generative Model by Fusion of 3D Information for De Novo Drug Design.

DrugGen: Advancing Drug Discovery with Large Language Models and Reinforcement Learning Feedback

MolGPT: Molecular Generation Using a Transformer-Decoder Model

Generating and screening de novo compounds against given targets using ultrafast deep learning models as core components

Drug-target interaction prediction using a multi-modal transformer network demonstrates high generalizability to unseen proteins

LigGPT: Molecular Generation using a Transformer-Decoder Model

Generative Pre-trained Transformer (GPT) based model with relative attention for de novo drug design

Target Specific De Novo Design of Drug Candidate Molecules with Graph Transformer-based Generative Adversarial Networks

Generative network complex (GNC) for drug discovery