Abstract:An algorithm is presented for the coupled-cluster singles, doubles, and perturbative triples correction [CCSD(T)] method based on the density fitting or the resolution-of-the-identity (RI) approximation for performing calculations on heterogeneous computing platforms composed of multicore CPUs and graphics processing units (GPUs). The directive-based approach to GPU offloading offered by the OpenMP application programming interface has been employed to adapt the most compute-intensive terms in the RI-CCSD amplitude equations with computational costs scaling as <math><mi>O</mi><mrow><mo>(</mo><msubsup><mi>N</mi><mi>O</mi><mn>2</mn></msubsup><msubsup><mi>N</mi><mi>V</mi><mn>4</mn></msubsup><mo>)</mo></mrow></math>, <math><mi>O</mi><mrow><mo>(</mo><msubsup><mi>N</mi><mi>O</mi><mn>3</mn></msubsup><msubsup><mi>N</mi><mi>V</mi><mn>3</mn></msubsup><mo>)</mo></mrow></math>, and <math><mi>O</mi><mrow><mo>(</mo><msubsup><mi>N</mi><mi>O</mi><mn>4</mn></msubsup><msubsup><mi>N</mi><mi>V</mi><mn>2</mn></msubsup><mo>)</mo></mrow></math> (where NO and NV denote the numbers of correlated occupied and virtual orbitals, respectively) and the perturbative triples correction to execute on GPU architectures. The pertinent tensor contractions are performed using an accelerated math library such as cuBLAS or hipBLAS. Optimal strategies are discussed for splitting large data arrays into tiles to fit them into the relatively small memory space of the GPUs, while also minimizing the low-bandwidth CPU-GPU data transfers. The performance of the hybrid CPU-GPU RI-CCSD(T) code is demonstrated on pre-exascale supercomputers composed of heterogeneous nodes equipped with NVIDIA Tesla V100 and A100 GPUs and on the world's first exascale supercomputer named "Frontier", the nodes of which consist of AMD MI250X GPUs. Speedups within the range 4-8× relative to the recently reported CPU-only algorithm are obtained for the GPU-offloaded terms in the RI-CCSD amplitude equations. Applications to polycyclic aromatic hydrocarbons containing 16-66 carbon atoms demonstrate that the acceleration of the hybrid CPU-GPU code for the perturbative triples correction relative to the CPU-only code increases with the molecule size, attaining a speedup of 5.7× for the largest circumovalene molecule (C66H20). The GPU-offloaded code enables the computation of the perturbative triples correction for the C60 molecule using the cc-pVDZ/aug-cc-pVTZ-RI basis sets in 7 min on Frontier when using 12,288 AMD GPUs with a parallel efficiency of 83.1%.

Generating coupled cluster code for modern distributed memory tensor software

Heterogeneous Programming and Optimization of Gyrokinetic Toroidal Code and Large-Scale Performance Test on TH-1A.

Accelerating Coupled-Cluster Calculations with GPUs: An Implementation of the Density-Fitted CCSD(T) Approach for Heterogeneous Computing Architectures Using OpenMP Directives

Optimizing Distributed Tensor Contractions using Node-Aware Processor Grids

Software for Sparse Tensor Decomposition on Emerging Computing Architectures

Accelerating Pythonic coupled cluster implementations: a comparison between CPUs and GPUs

Automatic transformation of irreducible representations for efficient contraction of tensors with cyclic group symmetry

Massive-Parallel Implementation of the Resolution-of-Identity Coupled-Cluster Approaches in the Numeric Atom-Centered Orbital Framework for Molecular Systems

COMET: A Domain-Specific Compilation of High-Performance Computational Chemistry

A Framework for Lattice QCD Calculations on GPUs

ACE: A general-purpose non-Markovian open quantum systems simulation toolkit based on process tensors

Yet Another Tensor Toolbox for discontinuous Galerkin methods and other applications

Sparse tensor based nuclear gradients for periodic Hartree-Fock and low-scaling correlated wave function methods in the CP2K software package: A massively parallel and GPU accelerated implementation

Accelerating Pythonic Coupled-Cluster Implementations: A Comparison Between CPUs and GPUs

A framework for load balancing of tensor contraction expressions via dynamic task partitioning

Polyhedral Specification and Code Generation of Sparse Tensor Contraction with Co-iteration

A GPGPU based program to solve the TDSE in intense laser fields through the finite difference approach

Accelerating Fortran Codes: A Method for Integrating Coarray Fortran with CUDA Fortran and OpenMP

PASCI : A Scalable Framework for Heterogeneous Parallel Calculation of Dynamical Electron Correlation

T2S-Tensor: Productively Generating High-Performance Spatial Hardware for Dense Tensor Computations