Abstract:An algorithm is presented for the coupled-cluster singles, doubles, and perturbative triples correction [CCSD(T)] method based on the density fitting or the resolution-of-the-identity (RI) approximation for performing calculations on heterogeneous computing platforms composed of multicore CPUs and graphics processing units (GPUs). The directive-based approach to GPU offloading offered by the OpenMP application programming interface has been employed to adapt the most compute-intensive terms in the RI-CCSD amplitude equations with computational costs scaling as <math><mi>O</mi><mrow><mo>(</mo><msubsup><mi>N</mi><mi>O</mi><mn>2</mn></msubsup><msubsup><mi>N</mi><mi>V</mi><mn>4</mn></msubsup><mo>)</mo></mrow></math>, <math><mi>O</mi><mrow><mo>(</mo><msubsup><mi>N</mi><mi>O</mi><mn>3</mn></msubsup><msubsup><mi>N</mi><mi>V</mi><mn>3</mn></msubsup><mo>)</mo></mrow></math>, and <math><mi>O</mi><mrow><mo>(</mo><msubsup><mi>N</mi><mi>O</mi><mn>4</mn></msubsup><msubsup><mi>N</mi><mi>V</mi><mn>2</mn></msubsup><mo>)</mo></mrow></math> (where NO and NV denote the numbers of correlated occupied and virtual orbitals, respectively) and the perturbative triples correction to execute on GPU architectures. The pertinent tensor contractions are performed using an accelerated math library such as cuBLAS or hipBLAS. Optimal strategies are discussed for splitting large data arrays into tiles to fit them into the relatively small memory space of the GPUs, while also minimizing the low-bandwidth CPU-GPU data transfers. The performance of the hybrid CPU-GPU RI-CCSD(T) code is demonstrated on pre-exascale supercomputers composed of heterogeneous nodes equipped with NVIDIA Tesla V100 and A100 GPUs and on the world's first exascale supercomputer named "Frontier", the nodes of which consist of AMD MI250X GPUs. Speedups within the range 4-8× relative to the recently reported CPU-only algorithm are obtained for the GPU-offloaded terms in the RI-CCSD amplitude equations. Applications to polycyclic aromatic hydrocarbons containing 16-66 carbon atoms demonstrate that the acceleration of the hybrid CPU-GPU code for the perturbative triples correction relative to the CPU-only code increases with the molecule size, attaining a speedup of 5.7× for the largest circumovalene molecule (C66H20). The GPU-offloaded code enables the computation of the perturbative triples correction for the C60 molecule using the cc-pVDZ/aug-cc-pVTZ-RI basis sets in 7 min on Frontier when using 12,288 AMD GPUs with a parallel efficiency of 83.1%.

Accelerating Pythonic Coupled-Cluster Implementations: A Comparison Between CPUs and GPUs

Accelerating Pythonic coupled cluster implementations: a comparison between CPUs and GPUs

Heterogeneous Programming and Optimization of Gyrokinetic Toroidal Code and Large-Scale Performance Test on TH-1A.

The Implementation of the Three-Dimensional Unified Gas-Kinetic Wave-Particle Method on Multiple Graphics Processing Units

Accelerating molecular dynamics simulations using Graphics Processing Units with CUDA

Accelerating atmospheric physics parameterizations using graphics processing units

Accelerating Coupled-Cluster Calculations with GPUs: An Implementation of the Density-Fitted CCSD(T) Approach for Heterogeneous Computing Architectures Using OpenMP Directives

On the Efficient Evaluation of the Exchange Correlation Potential on Graphics Processing Unit Clusters

Kernel Fusion in Atomistic Spin Dynamics Simulations on Nvidia GPUs using Tensor Core

Impact of CUDA and OpenCL on Parallel and Distributed Computing

High Performance Computing Via a GPU

Performance Analysis of CP2K Code for Ab Initio Molecular Dynamics on CPUs and GPUs

General-purpose molecular dynamics simulations on GPU-based clusters

Efficient molecular dynamics simulations with many-body potentials on graphics processing units

Performance Acceleration of Kernel Polynomial Method Applying Graphics Processing Units

PyPOD-GP: Using PyTorch for Accelerated Chip-Level Thermal Simulation of the GPU

A Parallel Sparse Tensor Benchmark Suite on CPUs and GPUs

Analyzing Molecular Simulations Trajectories by Utilizing CUDA on GPU Architecture

Kernel concurrency opportunities based on GPU benchmarks characterization

Accelerating Lattice QCD Simulations using GPUs

Performance Portable Monte Carlo Neutron Transport in MCDC via Numba