Abstract:De novo drug design refers to the process of designing new drug molecules from scratch using computational methods. In contrast to other computational methods that primarily focus on modifying existing molecules, designing from scratch enables the exploration of new chemical space and the potential discovery of novel molecules with enhanced properties. In this research, we proposed a model that utilizes Generative Pre-trained Transformer (GPT) architecture and relative attention for de novo drug design. GPT is a language model that utilizes transformer architecture to predict the next word or token in a given sequence. Representation of molecules using SMILES notation has enabled the use of next-token prediction techniques in de novo drug design. GPT uses attention mechanisms to capture the dependencies and relationships between different tokens in a sequence and allows the model to focus on the most important information when processing the input. Relative attention is a variant of the attention mechanism, which allows the model to capture the relative distances and relationships between tokens in the input sequence. In the standard attention mechanism, positional information is typically encoded using fixed-position embeddings. In relative attention, positional information is supplied dynamically during attention calculation by incorporating relative positional encodings, enabling the model to quickly learn the syntax of new unseen tokens. Relative attention enables the GPT model to better understand the relative positions of tokens in the sequence, which can be particularly useful when dealing with limited dataset sizes or generating target-specific drugs. The proposed model was trained on benchmark datasets, and performance was compared with other generative models. We show that relative attention and transfer learning could enable the GPT model to generate molecules with improved validity, uniqueness, and novelty in the context of de novo drug design. To illustrate the effectiveness of relative attention, the model was trained using transfer learning on three target-specific datasets, and the performance was compared with standard attention.

GPT like transformer based conditional molecule generator and a high drug-likeness (QED) dataset generation

PrefixMol: Target- and Chemistry-aware Molecule Design Via Prefix Embedding

cMolGPT: A Conditional Generative Pre-Trained Transformer for Target-Specific De Novo Molecular Generation

TD-GPT: Target Protein-Specific Drug Molecule Generation GPT

Adapt-cMolGPT: A Conditional Generative Pre-Trained Transformer with Adapter-Based Fine-Tuning for Target-Specific Molecular Generation

3DSMILES-GPT: 3D Molecular Pocket-based Generation with Token-only Large Language Model

MolGPT: Molecular Generation Using a Transformer-Decoder Model

Diffusing on Two Levels and Optimizing for Multiple Properties: A Novel Approach to Generating Molecules with Desirable Properties

NPGPT: Natural Product-Like Compound Generation with GPT-based Chemical Language Models

GexMolGen: Cross-modal Generation of Hit-like Molecules via Large Language Model Encoding of Gene Expression Signatures

3D Molecular Pocket-based Generation with Token-only Large Language Model

RM-GPT: Enhance the comprehensive generative ability of molecular GPT model via LocalRNN and RealFormer

MedGAN: optimized generative adversarial network with graph convolutional networks for novel molecule design

CMGN: a conditional molecular generation net to design target-specific molecules with desired properties

LigGPT: Molecular Generation using a Transformer-Decoder Model

Probabilistic generative transformer language models for generative design of molecules

A Transformer-based Generative Model for De Novo Molecular Design

Generative Pre-trained Transformer (GPT) based model with relative attention for de novo drug design

Quantum Generative Models for Small Molecule Drug Discovery

DockingGA: enhancing targeted molecule generation using transformer neural network and genetic algorithm with docking simulation