Abstract:Recently, artificial intelligence for drug discovery has raised increasing interest in both machine learning and chemistry domains. The fundamental building block for drug discovery is molecule geometry and thus, the molecule's geometrical representation is the main bottleneck to better utilize machine learning techniques for drug discovery. In this work, we propose a pretraining method for molecule joint auto-encoding (MoleculeJAE). MoleculeJAE can learn both the 2D bond (topology) and 3D conformation (geometry) information, and a diffusion process model is applied to mimic the augmented trajectories of such two modalities, based on which, MoleculeJAE will learn the inherent chemical structure in a self-supervised manner. Thus, the pretrained geometrical representation in MoleculeJAE is expected to benefit downstream geometry-related tasks. Empirically, MoleculeJAE proves its effectiveness by reaching state-of-the-art performance on 15 out of 20 tasks by comparing it with 12 competitive baselines.

What problem does this paper attempt to address?

This paper primarily addresses a key issue in molecular representation learning: how to effectively utilize machine learning techniques to handle molecular geometric information in drug discovery. Specifically, the paper proposes MoleculeJAE (Molecule Joint Auto-Encoding), a novel pre-training method designed to jointly encode the 2D topological structure (chemical bond structure) and 3D conformational structure (geometric structure) of molecules, and enhance the performance of downstream tasks through self-supervised learning. ### Main Issues Addressed 1. **Bottleneck of Molecular Geometric Representation**: In drug discovery, molecular geometric representation is fundamental, but existing machine learning techniques face challenges in handling complex molecular data, especially in effectively utilizing large-scale unlabeled datasets for molecular property prediction and molecular generation. 2. **Representation Learning of Complex Graph Structures**: Molecular data has complex graph structures and symmetries, making it difficult to learn powerful representations from such data. Existing methods like generative models and contrastive models have limitations in handling these complex structures. 3. **Combining 2D and 3D Information**: Previous works typically handle the 2D topological structure or 3D conformational structure of molecules independently, failing to combine the two effectively. The paper attempts to improve representation learning by jointly encoding information from these two modalities. ### Core Contributions - Proposes the MoleculeJAE framework, which can simultaneously learn the 2D topological structure and 3D conformational structure of molecules, and uses a diffusion process model to simulate the enhancement trajectories of these two modalities, thereby learning the intrinsic chemical structure of molecules in a self-supervised manner based on these trajectories. - The paper also explores how to leverage the deep connection between the forward process of the diffusion model and data augmentation, and how to capture the association between the true data distribution and its noisy counterparts through joint distribution fitting. - Experimental results show that MoleculeJAE outperforms other baseline methods in multiple downstream tasks (including energy prediction of stable conformations and force prediction along molecular dynamics), achieving significant results particularly in quantum mechanics prediction tasks and force prediction tasks on the QM9 and MD17 datasets. In summary, this research aims to address key bottleneck issues in molecular representation learning and effectively enhance the performance of downstream tasks by proposing a novel self-supervised pre-training framework, MoleculeJAE.

Molecule Joint Auto-Encoding: Trajectory Pretraining with 2D and 3D Diffusion

A Group Symmetric Stochastic Differential Equation Model for Molecule Multi-modal Pretraining

Learning Joint 2D & 3D Diffusion Models for Complete Molecule Generation

Learning Joint 2-D and 3-D Graph Diffusion Models for Complete Molecule Generation

Automated 3D Pre-Training for Molecular Property Prediction

3D-Transformer: Molecular Representation with Transformer in 3D Space

Unified 2D and 3D Pre-Training of Molecular Representations

MOL-AE: Auto-Encoder Based Molecular Representation Learning With 3D Cloze Test Objective

Diffusion-Driven Generative Framework for Molecular Conformation Prediction

3D-Mol: A Novel Contrastive Learning Framework for Molecular Property Prediction with 3D Information

Dual-view Molecular Pre-training

Pre-training Molecular Graph Representation with 3D Geometry

Quantum-Informed Molecular Representation Learning Enhancing ADMET Property Prediction

3D Equivariant Diffusion for Target-Aware Molecule Generation and Affinity Prediction

Molecular CT: Unifying Geometry and Representation Learning for Molecules at Different Scales

Pretraining Graph Transformer for Molecular Representation with Fusion of Multimodal Information

GTAM: A Molecular Pretraining Model with Geometric Triangle Awareness

Leveraging 2D molecular graph pretraining for improved 3D conformer generation with graph neural networks

GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generation