Abstract:Atomic structure prediction and associated property calculations are the bedrock of chemical physics. Since high-fidelity ab initio modeling techniques for computing the structure and properties can be prohibitively expensive, this motivates the development of machine-learning (ML) models that make these predictions more efficiently. Training graph neural networks over large atomistic databases introduces unique computational challenges, such as the need to process millions of small graphs with variable size and support communication patterns that are distinct from learning over large graphs, such as social networks. We demonstrate a novel hardware-software codesign approach to scale up the training of atomistic graph neural networks (GNN) for structure and property prediction. First, to eliminate redundant computation and memory associated with alternative padding techniques and to improve throughput via minimizing communication, we formulate the effective coalescing of the batches of variable-size atomistic graphs as the bin packing problem and introduce a hardware-agnostic algorithm to pack these batches. In addition, we propose hardware-specific optimizations, including a planner and vectorization for the gather-scatter operations targeted for Graphcore's Intelligence Processing Unit (IPU), as well as model-specific optimizations such as merged communication collectives and optimized softplus. Putting these all together, we demonstrate the effectiveness of the proposed codesign approach by providing an implementation of a well-established atomistic GNN on the Graphcore IPUs. We evaluate the training performance on multiple atomistic graph databases with varying degrees of graph counts, sizes, and sparsity. We demonstrate that such a codesign approach can reduce the training time of atomistic GNNs and can improve their performance by up to 1.5× compared to the baseline implementation of the model on the IPUs. Additionally, we compare our IPU implementation with a Nvidia GPU-based implementation and show that our atomistic GNN implementation on the IPUs can run 1.8× faster on average compared to the execution time on the GPUs.

Scalable Parallel Algorithm for Graph Neural Network Interatomic Potentials in Molecular Dynamics Simulations

Scalable Parallel Algorithm for Graph Neural Network Interatomic Potentials in Molecular Dynamics Simulations

A Scalable Graph Neural Network Method For Developing An Accurate Force Field Of Large Flexible Organic Molecules

Accurate and scalable graph neural network force field and molecular dynamics with direct force architecture

Accurate and scalable multi-element graph neural network force field and molecular dynamics with direct force architecture

How Graph Neural Network Interatomic Potentials Extrapolate: Role of the Message-Passing Algorithm

Towards Training Billion Parameter Graph Neural Networks for Atomic Simulations

The Importance of Being Scalable: Improving the Speed and Accuracy of Neural Network Interatomic Potentials Across Chemical Domains

NeuraChip: Accelerating GNN Computations with a Hash-based Decoupled Spatial Accelerator

Acceleration of Graph Neural Network-Based Prediction Models in Chemistry via Co-Design Optimization on Intelligence Processing Units

Parallel and Distributed Graph Neural Networks: An In-Depth Concurrency Analysis

Scalable training of graph convolutional neural networks for fast and accurate predictions of HOMO-LUMO gap in molecules

GSplit: Scaling Graph Neural Network Training on Large Graphs via Split-Parallelism

Long-Short-Range Message-Passing: A Physics-Informed Framework to Capture Non-Local Interaction for Scalable Molecular Dynamics Simulation

Simulate Time-integrated Coarse-grained Molecular Dynamics with Multi-Scale Graph Networks

On the Scalability of GNNs for Molecular Graphs

Scalable and Efficient Full-Graph GNN Training for Large Graphs

Graph-EAM: an Interpretable and Efficient Graph Neural Network Potential Framework.

Scalable and Consistent Graph Neural Networks for Distributed Mesh-based Data-driven Modeling

Analyzing the Performance of Graph Neural Networks with Pipe Parallelism

GPU-accelerated artificial neural network potential for molecular dynamics simulation