Abstract:We propose a combination of a variational autoencoder and a transformer based model which fully utilises graph convolutional and graph pooling layers to operate directly on graphs. The transformer model implements a novel node encoding layer, replacing the position encoding typically used in transformers, to create a transformer with no position information that operates on graphs, encoding adjacent node properties into the edge generation process. The proposed model builds on graph generative work operating on graphs with edge features, creating a model that offers improved scalability with the number of nodes in a graph. In addition, our model is capable of learning a disentangled, interpretable latent space that represents graph properties through a mapping between latent variables and graph properties. In experiments we chose a benchmark task of molecular generation, given the importance of both generated node and edge features. Using the QM9 dataset we demonstrate that our model performs strongly across the task of generating valid, unique and novel molecules. Finally, we demonstrate that the model is interpretable by generating molecules controlled by molecular properties, and we then analyse and visualise the learned latent representation.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to effectively generate new molecular graphs with desired properties in the molecular graph generation task. Specifically, the paper proposes a model that combines Variational Auto - Encoder (VAE) and Transformer, which can operate directly on the graph structure and use graph convolution and graph pooling layers to generate molecular graphs. This method aims to overcome the limitations of existing methods when dealing with large - scale molecular graphs, such as poor scalability and insufficient effectiveness of generated molecules. ### Main Contributions 1. **Model Architecture**: Proposed a model that combines Variational Auto - Encoder (VAE) and Transformer, which can operate directly on the graph structure, use graph convolution and graph pooling layers to generate node features, and predict edge features through the Transformer model. 2. **Node Encoding**: Introduced a new node encoding layer to replace the position encoding in the traditional Transformer, enabling the model to operate on the graph structure without position information and encode the properties of adjacent nodes during the edge prediction process. 3. **Decoupled Latent Space**: By establishing a mapping from latent variables to graph properties in the latent space, the control of the molecular graph generation process is achieved, making the generated molecules have specific quantum properties. 4. **Performance Evaluation**: Experiments were carried out on the QM9 dataset, demonstrating the strong performance of the model in generating valid, unique, and novel molecules. ### Specific Problems - **Generating Valid Molecules**: Ensure that the generated molecules are chemically valid, that is, they can be parsed into valid SMILES strings. - **Generating Unique Molecules**: Ensure that the generated molecules are unique in the dataset and avoid repeatedly generating the same molecules. - **Generating Novel Molecules**: Ensure that the generated molecules are new molecules that do not exist in the dataset. - **Interpretability and Controllability**: Through the decoupled latent space, control over specific properties of the generated molecules, such as dipole moment, zero - point vibrational energy, etc., is achieved. ### Experimental Results - **Validity**: 74.6% of the generated molecules are valid. - **Uniqueness**: 22.5% of the generated molecules are unique. - **Novelty**: 93.9% of the generated molecules are novel. - **Comprehensive Performance**: In the comprehensive index of generating valid, unique, and novel molecules, this model reaches 15.8%, outperforming other benchmark models. ### Conclusion The model proposed in this paper performs excellently in the molecular graph generation task. It not only achieves good performance in generating valid, unique, and novel molecules but also realizes the control of the generation process through the decoupled latent space, providing a powerful tool for designing new molecules with specific properties. Future work will further develop graph pooling techniques to support the generation of larger - scale molecular graphs and construct datasets containing larger molecules to meet the needs of drug discovery and new material development.

A Graph VAE and Graph Transformer Approach to Generating Molecular Graphs

Transformer Graph Variational Autoencoder for Generative Molecular Design

MoVAE: A Variational AutoEncoder for Molecular Graph Generation.

A novel molecule generative model of VAE combined with Transformer for unseen structure generation

Disentangle VAE for Molecular Generation

GraphVAE: Towards Generation of Small Graphs Using Variational Autoencoders

Autoencoding Undirected Molecular Graphs With Neural Networks

VGAE-MCTS: A New Molecular Generative Model Combining the Variational Graph Auto-Encoder and Monte Carlo Tree Search

Leveraging Tree-Transformer VAE with fragment tokenization for high-performance large chemical generative model

RGCVAE: Relational Graph Conditioned Variational Autoencoder for Molecule Design

A Tree-Transformer based VAE with fragment tokenization for large chemical models

Fragment‐based Deep Molecular Generation using Hierarchical Chemical Graph Representation and Multi‐Resolution Graph Variational Autoencoder

GraphGPT: A Graph Enhanced Generative Pretrained Transformer for Conditioned Molecular Generation

Graph Embedding VAE: A Permutation Invariant Model of Graph Structure

GraphXForm: Graph transformer for computer-aided molecular design with application to extraction

DEMO: Disentangled Molecular Graph Generation via an Invertible Flow Model

3D-Transformer: Molecular Representation with Transformer in 3D Space

Decoding Molecular Graph Embeddings with Reinforcement Learning

Probabilistic generative transformer language models for generative design of molecules

Graph-Aware Transformer: Is Attention All Graphs Need?

GraphPiece: Efficiently Generating High-Quality Molecular Graph with Substructures