Abstract:Spatial structures in the 3D space are important to determine molecular properties. Recent papers use geometric deep learning to represent molecules and predict properties. These papers, however, are computationally expensive in capturing long-range dependencies of input atoms; and more importantly, they have not considered the non-uniformity of interatomic distances, thus failing to learn context-dependent representations at different scales. To deal with such issues, we introduce 3D-Transformer, a variant of the Transformer for molecular representations that incorporates 3D spatial information. 3D-Transformer operates on a fully-connected graph with direct connections between atoms. To cope with the non-uniformity of interatomic distances, we develop a multi-scale self-attention module that exploits local fine-grained patterns with increasing contextual scales. As molecules of different sizes rely on different kinds of spatial features, we design an adaptive position encoding module that adopts different position encoding methods for small and large molecules. Finally, to attain the molecular representation from atom embeddings, we propose an attentive farthest point sampling algorithm that selects a portion of atoms with the assistance of attention scores, overcoming handicaps of the virtual node and previous distance-dominant downsampling methods. We validate 3D-Transformer across three important scientific domains: quantum chemistry, material science,and proteomics. Our experiments show significant improvements over state-of-the-art models on the crystal property prediction task and the protein-ligand binding affinity prediction task, and show better or competitive performance in quantum chemistry molecular datasets. This work provides clear evidence that biochemical tasks can gain consistent benefits from 3D molecular representations and different tasks require different position encoding methods.

Employing Artificial Neural Networks for Optimal Storage and Facile Sharing of Molecular Dynamics Simulation Trajectories

Compression of Uncertain Trajectories in Road Networks.

Parallel Performance of Molecular Dynamics Trajectory Analysis

Generative Modeling of Molecular Dynamics Trajectories

3D-Transformer: Molecular Representation with Transformer in 3D Space

Efficient Compression of Sparse Accelerator Data Using Implicit Neural Representations and Importance Sampling

Scalable Hybrid Learning Techniques for Scientific Data Compression

Analyzing Molecular Simulations Trajectories by Utilizing CUDA on GPU Architecture

EM-Compressor: Electron Microscopy Image Compression in Connectomics with Variational Autoencoders

A Deep Graph Network–Enhanced Sampling Approach to Efficiently Explore the Space of Reduced Representations of Proteins

A novel genetic algorithm-based approach for compression and acceleration of deep learning convolution neural network: an application in computer tomography lung cancer data

Deep DNA Storage: Scalable and Robust DNA Storage via Coding Theory and Deep Learning

Learning Geometrically Disentangled Representations of Protein Folding Simulations

Dynamic Molecular Graph-based Implementation for Biophysical Properties Prediction

Running and analyzing massively parallel molecular simulations

Implicit Neural Multiple Description for DNA-based data storage

DAVT: an Error-Bounded Vehicle Trajectory Data Representation and Compression Framework

Evaluating the effect of compressing algorithms for trajectory similarity and classification problems

A cloud platform for automating and sharing analysis of raw simulation data from high throughput polymer molecular dynamics simulations

DiffMD: A Geometric Diffusion Model for Molecular Dynamics Simulations

TorchMD-Net 2.0: Fast Neural Network Potentials for Molecular Simulations