Abstract:Recently, molecule generation using deep learning has been actively investigated in drug discovery. In this field, Transformer and VAE are widely used as powerful models, but they are rarely used in combination due to structural and performance mismatch of them. This study proposes a model that combines these two models through structural and parameter optimization in handling diverse molecules. The proposed model shows comparable performance to existing models in generating molecules, and showed by far superior performance in generating molecules with unseen structures. Another advantage of this VAE model is that it generates molecules from latent representation, and therefore properties of molecules can be easily predicted or conditioned with it, and indeed, we show that the latent representation of the model successfully predicts molecular properties. Ablation study suggested the advantage of VAE over other generative models like language model in generating novel molecules. It also indicated that the latent representation can be shortened to ~32 dimensional variables without loss of reconstruction, suggesting the possibility of a much smaller molecular descriptor or model than existing ones. This study is expected to provide a virtual chemical library containing a wide variety of compounds for virtual screening and to enable efficient screening.

What problem does this paper attempt to address?

The paper aims to address the problem of molecular generation in the field of drug discovery, specifically how to use deep learning techniques to generate novel small molecules with potential medicinal value. Specifically, the research objectives include: 1. **Combining Variational Autoencoder (VAE) with Transformer Model**: Although VAE and Transformer are both powerful deep learning models and have been widely used in molecular generation tasks, they are rarely combined. This is due to structural and performance mismatches between them. This paper proposes a method to optimize the structure and parameters to successfully combine these two models. 2. **Generating Novel Molecular Structures**: The focus of the research is on generating molecular structures that have not appeared in the training dataset. This means that the model needs to be able to explore unexplored parts of the chemical space, which is crucial for new drug discovery. 3. **Maintaining Molecular Property Prediction Capability**: Since VAE can convert molecules into latent representations, it becomes easier to predict or adjust the properties of the generated molecules. This paper demonstrates that the latent representations of the model can successfully predict molecular properties and have advantages in generating molecules with desired characteristics. 4. **Optimizing Latent Space Dimensions**: The research also explores the dimensions of the latent variable space, finding that even reducing it to 32 dimensions can maintain good reconstruction performance. This indicates that molecules can be represented with more concise descriptors. Through the above research, the model not only shows excellent performance in generating molecules but also exhibits significant advantages in generating unseen molecular structures. This is of great significance for expanding virtual chemical libraries and improving virtual screening efficiency.

A novel molecule generative model of VAE combined with Transformer for unseen structure generation

Transformer Graph Variational Autoencoder for Generative Molecular Design

Leveraging Tree-Transformer VAE with fragment tokenization for high-performance large chemical generative model

A Graph VAE and Graph Transformer Approach to Generating Molecular Graphs

Disentangle VAE for Molecular Generation

VGAE-MCTS: A New Molecular Generative Model Combining the Variational Graph Auto-Encoder and Monte Carlo Tree Search

MoVAE: A Variational AutoEncoder for Molecular Graph Generation.

A Tree-Transformer based VAE with fragment tokenization for large chemical models

Molecular substructure tree generative model for de novo drug design

Molecular Generative Model Based on an Adversarially Regularized Autoencoder

Probabilistic generative transformer language models for generative design of molecules

Molecular Generative Model Based On Adversarially Regularized Autoencoder

Conditional $β$-VAE for De Novo Molecular Generation

Fragment‐based Deep Molecular Generation using Hierarchical Chemical Graph Representation and Multi‐Resolution Graph Variational Autoencoder

Objective-Agnostic Enhancement of Molecule Properties via Multi-Stage VAE

A Transformer-based Generative Model for De Novo Molecular Design

Developing a Generative Model Utilizing Self‐attention Networks: Application to Materials/Drug Discovery

Sc2Mol: a Scaffold-Based Two-Step Molecule Generator with Variational Autoencoder and Transformer.

Nc-vae: normalised conditional diverse variational autoencoder guided de novo molecule generation

Generative Model for Small Molecules with Latent Space RL Fine-Tuning to Protein Targets

Molecule Sequence Generation with Rebalanced Variational Autoencoder Loss