Abstract:Spatial structures in the 3D space are important to determine molecular properties. Recent papers use geometric deep learning to represent molecules and predict properties. These papers, however, are computationally expensive in capturing long-range dependencies of input atoms; and more importantly, they have not considered the non-uniformity of interatomic distances, thus failing to learn context-dependent representations at different scales. To deal with such issues, we introduce 3D-Transformer, a variant of the Transformer for molecular representations that incorporates 3D spatial information. 3D-Transformer operates on a fully-connected graph with direct connections between atoms. To cope with the non-uniformity of interatomic distances, we develop a multi-scale self-attention module that exploits local fine-grained patterns with increasing contextual scales. As molecules of different sizes rely on different kinds of spatial features, we design an adaptive position encoding module that adopts different position encoding methods for small and large molecules. Finally, to attain the molecular representation from atom embeddings, we propose an attentive farthest point sampling algorithm that selects a portion of atoms with the assistance of attention scores, overcoming handicaps of the virtual node and previous distance-dominant downsampling methods. We validate 3D-Transformer across three important scientific domains: quantum chemistry, material science,and proteomics. Our experiments show significant improvements over state-of-the-art models on the crystal property prediction task and the protein-ligand binding affinity prediction task, and show better or competitive performance in quantum chemistry molecular datasets. This work provides clear evidence that biochemical tasks can gain consistent benefits from 3D molecular representations and different tasks require different position encoding methods.

Tandem mass spectrum prediction for small molecules using graph transformers

Emergence of molecular structures from repository-scale self-supervised learning on tandem mass spectra

Prefix-Tree Decoding for Predicting Mass Spectra from Molecules

Enhancing Molecular Structure Elucidation: MultiModalTransformer for both simulated and experimental spectra

TransG-net: transformer and graph neural network based multi-modal data fusion network for molecular properties prediction

Structural Annotation of Unknown Molecular for Miniaturized Mass Spectrometer based on Transformer Enabled Fragment Tree

Structural annotation of unknown molecules in a miniaturized mass spectrometer based on a transformer enabled fragment tree method

Mass Spectra Prediction with Structural Motif-based Graph Neural Networks

FraGNNet: A Deep Probabilistic Model for Mass Spectrum Prediction

Advancing the Prediction of MS/MS Spectra Using Machine Learning

3D-Transformer: Molecular Representation with Transformer in 3D Space

LSM1-MS2: A Self-Supervised Foundation Model for Tandem Mass Spectrometry Applications, Encompassing Extensive Chemical Property Predictions and Spectral Matching

Using Graph Neural Networks for Mass Spectrometry Prediction

Contrastive Learning-Based Embedder for the Representation of Tandem Mass Spectra

MIST-CF: Chemical formula inference from tandem mass spectra

Generating Molecular Fragmentation Graphs with Autoregressive Neural Networks

MSBERT: Embedding Tandem Mass Spectra into Chemically Rational Space by Mask Learning and Contrastive Learning

Pre-trained Maldi Transformers improve MALDI-TOF MS-based prediction

LSM1-MS2: A Foundation Model for MS/MS, Encompassing Chemical Property Predictions, Search and de novo Generation