Abstract:Spatial structures in the 3D space are important to determine molecular properties. Recent papers use geometric deep learning to represent molecules and predict properties. These papers, however, are computationally expensive in capturing long-range dependencies of input atoms; and more importantly, they have not considered the non-uniformity of interatomic distances, thus failing to learn context-dependent representations at different scales. To deal with such issues, we introduce 3D-Transformer, a variant of the Transformer for molecular representations that incorporates 3D spatial information. 3D-Transformer operates on a fully-connected graph with direct connections between atoms. To cope with the non-uniformity of interatomic distances, we develop a multi-scale self-attention module that exploits local fine-grained patterns with increasing contextual scales. As molecules of different sizes rely on different kinds of spatial features, we design an adaptive position encoding module that adopts different position encoding methods for small and large molecules. Finally, to attain the molecular representation from atom embeddings, we propose an attentive farthest point sampling algorithm that selects a portion of atoms with the assistance of attention scores, overcoming handicaps of the virtual node and previous distance-dominant downsampling methods. We validate 3D-Transformer across three important scientific domains: quantum chemistry, material science,and proteomics. Our experiments show significant improvements over state-of-the-art models on the crystal property prediction task and the protein-ligand binding affinity prediction task, and show better or competitive performance in quantum chemistry molecular datasets. This work provides clear evidence that biochemical tasks can gain consistent benefits from 3D molecular representations and different tasks require different position encoding methods.

3D Computer Vision Models Predict DFT-Level HOMO-LUMO Gap Energies from Force-Field-Optimized Geometries

3D-Transformer: Molecular Representation with Transformer in 3D Space

Molecular Geometry Impact on Deep Learning Predictions of Inverted Singlet–Triplet Gaps

MolE8: Finding DFT Potential Energy Surface Minima Values from Force-Field Optimised Organic Molecules with New Machine Learning Representations

Molecule3D: A Benchmark for Predicting 3D Geometries from Molecular Graphs

3D Molecular Geometry Analysis with 2D Graphs

Quantum-chemical insights from deep tensor neural networks

Carbon-based molecular properties efficiently predicted by deep learning-based quantum chemical simulation with large language models

Quantum 3D graph structure learning with applications to molecule computing

Discrete Fourier Transform Improves the Prediction of the Electronic Properties of Molecules in Quantum Machine Learning

Non-equilibrium molecular geometries in graph neural networks

Symmetry-invariant quantum machine learning force fields

May the Force be with You: Unified Force-Centric Pre-Training for 3D Molecular Conformations

Resolution-vs.-Accuracy Dilemma in Machine Learning Modeling of Electronic Excitation Spectra

Assessing Conformation Validity and Rationality of Deep Learning-Generated 3D Molecules

Prediction of Energetic Material Properties from Electronic Structure Using 3D Convolutional Neural Networks

BAND NN: A Deep Learning Framework for Energy Prediction and Geometry Optimization of Organic Small Molecules

Quantum 3D Graph Learning with Applications to Molecule Embedding.

Semiempirical Hamiltonians learned from data can have accuracy comparable to Density Functional Theory

Deep Spatial Learning with Molecular Vibration

Investigating Data Hierarchies in Multifidelity Machine Learning for Excitation Energies