3D Molecular Pretraining via Localized Geometric Generation

Yuancheng Sun,Kai Chen,Kang Liu,Qiwei Ye

DOI: https://doi.org/10.1101/2024.09.10.612249

2024-09-14

Abstract:Self-supervised learning on 3D molecular structures is gaining importance in data-driven scientific research and applications due to the high costs of annotating biochemical data. However, the strategic selection of semantic units for modeling 3D molecular structures remains underexplored, despite its crucial role in effective pretraining - a concept well-established in language processing and computer vision. We introduce Localized Geometric Generation (LEGO), a novel approach that treats tetrahedrons within 3D molecular structures as fundamental building blocks, leveraging their geometric simplicity and widespread presence across chemical functional patterns. Inspired by masked modeling, LEGO perturbs tetrahedral local structures and learns to reconstruct them in a self-supervised manner. Experimental results demonstrate LEGO consistently enhances molecular representations across biochemistry and quantum property prediction benchmarks. Additionally, the tetrahedral modeling and pretraining generalize from small molecules to larger molecular systems, validating by protein-ligand affinity prediction. Our results highlight the potential of selecting semantic units to build more expressive and interpretable neural networks for scientific AI applications.

Bioinformatics

What problem does this paper attempt to address?

The paper attempts to address the issue of improving the effectiveness and generalization ability of molecular representations in 3D molecular structure modeling through self-supervised learning methods. Specifically:

1. **Selecting Appropriate Semantic Units**: Existing 3D molecular pre-training methods lack sufficient research on the selection of basic semantic units for modeling. The paper proposes a new approach that uses tetrahedrons as the basic building blocks in 3D molecular structures and designs pre-training tasks based on these tetrahedrons.

2. **Enhancing Molecular Representation Learning**: By utilizing self-supervised learning techniques, the method captures intrinsic patterns from a large amount of unlabeled molecular data and obtains meaningful representations, thereby improving the performance of downstream tasks.

3. **Improving Generalization Ability**: The paper verifies whether the proposed pre-training method has better generalization ability from smaller molecular systems to larger molecular systems (such as protein-ligand binding affinity prediction).

Through the above points, the paper aims to improve 3D molecular structure modeling and representation learning by introducing a new self-supervised pre-training method—Localized Geometric Generation (LEGO), thereby achieving better performance in tasks such as biochemical and quantum property prediction.