Molecular Graph Transformer: Stepping Beyond ALIGNN Into Long-Range Interactions

Devis Di Tommaso,Marco Anselmi,Rachel Crespo-Otero,Greg Slabaugh

DOI: https://doi.org/10.26434/chemrxiv-2024-rxwbc

2024-01-22

Abstract:Graph Neural Networks (GNNs) have revolutionized material property prediction by learning directly from the structural information of molecules and materials. However, conventional GNN models rely solely on local atomic interactions, such as bond lengths and angles, neglecting crucial longrange electrostatic forces that aect certain properties. To address this, we introduce the Molecular Graph Transformer (MGT), a novel GNN architecture that combines local attention mechanisms with message passing on both bond graphs and their line graphs, explicitly capturing long-range interactions. Benchmarking on MatBench and Quantum MOF (QMOF) datasets demonstrates that MGT's improved understanding of electrostatic interactions signicantly enhances the prediction accuracy of properties like exfoliation energy and refractive index, while maintaining state-of-theart performance on all other properties. This breakthrough paves the way for the development of highly accurate and efficient materials design tools across diverse applications. Code is available at: https://github.com/MolecularGraphTransformer/MGT

Chemistry

What problem does this paper attempt to address?

The main focus of this paper is on improving the ability of Graph Neural Networks (GNN) in predicting material properties, especially in handling long-range interactions between molecules. Current GNN models typically only consider local atomic interactions, such as bond lengths and bond angles, while neglecting crucial long-range forces such as electrostatic interactions, which are essential for predicting certain properties. To address this issue, the paper proposes a "Molecular Graph Transformer" (MGT). MGT explicitly captures long-range interactions by combining local attention mechanisms and information propagation based on bond graphs and their line graphs. It represents molecules using three subgraphs: the local graph, line graph, and fully connected graph, which are used to represent direct contacts between atoms, three-body interactions, and non-bonded interactions, respectively. MGT alternates between global graph attention layers and line graph convolution layers to update the molecular representation, leveraging non-bonded information, many-body information, and pairwise information. The paper conducts benchmark tests on the MatBench and Quantum MOF datasets, and the results show that MGT significantly improves accuracy in predicting properties such as exfoliation energy and refractive index, while maintaining state-of-the-art performance on all other properties. This breakthrough paves the way for the development of efficient and accurate material design tools, applicable to various applications. The code is publicly available on GitHub. The main contribution of the paper lies in the MGT architecture, which enhances the understanding of GNNs on long-range interactions, especially electrostatic interactions, and improves the accuracy of material property prediction.

Molecular Graph Transformer: Stepping Beyond ALIGNN Into Long-Range Interactions

Describe Molecules by a Heterogeneous Graph Neural Network with Transformer-like Attention for Supervised Property Predictions

Dynamic Molecular Graph-based Implementation for Biophysical Properties Prediction

ABT-MPNN: an atom-bond transformer-based message-passing neural network for molecular property prediction

Cross-dependent graph neural networks for molecular property prediction

Enhancing Molecular Representations Via Graph Transformation Layers

GT-NMR: a novel graph transformer-based approach for accurate prediction of NMR chemical shifts

Molecular Mechanics-Driven Graph Neural Network with Multiplex Graph for Molecular Structures

Multi-View Graph Neural Networks for Molecular Property Prediction

Neural Atoms: Propagating Long-range Interaction in Molecular Graphs through Efficient Communication Channel

3D-Transformer: Molecular Representation with Transformer in 3D Space

Long-Range Neural Atom Learning for Molecular Graphs.

Pretraining Graph Transformer for Molecular Representation with Fusion of Multimodal Information

TransG-net: transformer and graph neural network based multi-modal data fusion network for molecular properties prediction

Graph neural networks for the prediction of molecular structure-property relationships

Enhancing property and activity prediction and interpretation using multiple molecular graph representations with MMGX

Heterogeneous Molecular Graph Neural Networks for Predicting Molecule Properties

GMPP-NN: a deep learning architecture for graph molecular property prediction

Triplet Interaction Improves Graph Transformers: Accurate Molecular Graph Learning with Triplet Graph Transformers

Graph Neural Network Architecture Search for Molecular Property Prediction

Directed graph attention neural network utilizing 3D coordinates for molecular property prediction