Abstract:An algorithm is presented for the coupled-cluster singles, doubles, and perturbative triples correction [CCSD(T)] method based on the density fitting or the resolution-of-the-identity (RI) approximation for performing calculations on heterogeneous computing platforms composed of multicore CPUs and graphics processing units (GPUs). The directive-based approach to GPU offloading offered by the OpenMP application programming interface has been employed to adapt the most compute-intensive terms in the RI-CCSD amplitude equations with computational costs scaling as <math><mi>O</mi><mrow><mo>(</mo><msubsup><mi>N</mi><mi>O</mi><mn>2</mn></msubsup><msubsup><mi>N</mi><mi>V</mi><mn>4</mn></msubsup><mo>)</mo></mrow></math>, <math><mi>O</mi><mrow><mo>(</mo><msubsup><mi>N</mi><mi>O</mi><mn>3</mn></msubsup><msubsup><mi>N</mi><mi>V</mi><mn>3</mn></msubsup><mo>)</mo></mrow></math>, and <math><mi>O</mi><mrow><mo>(</mo><msubsup><mi>N</mi><mi>O</mi><mn>4</mn></msubsup><msubsup><mi>N</mi><mi>V</mi><mn>2</mn></msubsup><mo>)</mo></mrow></math> (where NO and NV denote the numbers of correlated occupied and virtual orbitals, respectively) and the perturbative triples correction to execute on GPU architectures. The pertinent tensor contractions are performed using an accelerated math library such as cuBLAS or hipBLAS. Optimal strategies are discussed for splitting large data arrays into tiles to fit them into the relatively small memory space of the GPUs, while also minimizing the low-bandwidth CPU-GPU data transfers. The performance of the hybrid CPU-GPU RI-CCSD(T) code is demonstrated on pre-exascale supercomputers composed of heterogeneous nodes equipped with NVIDIA Tesla V100 and A100 GPUs and on the world's first exascale supercomputer named "Frontier", the nodes of which consist of AMD MI250X GPUs. Speedups within the range 4-8× relative to the recently reported CPU-only algorithm are obtained for the GPU-offloaded terms in the RI-CCSD amplitude equations. Applications to polycyclic aromatic hydrocarbons containing 16-66 carbon atoms demonstrate that the acceleration of the hybrid CPU-GPU code for the perturbative triples correction relative to the CPU-only code increases with the molecule size, attaining a speedup of 5.7× for the largest circumovalene molecule (C66H20). The GPU-offloaded code enables the computation of the perturbative triples correction for the C60 molecule using the cc-pVDZ/aug-cc-pVTZ-RI basis sets in 7 min on Frontier when using 12,288 AMD GPUs with a parallel efficiency of 83.1%.

Treecode-accelerated Green Iteration for Kohn-Sham Density Functional Theory

Very-Large-Scale GPU-Accelerated Nuclear Gradient of Time-Dependent Density Functional Theory with Tamm-Dancoff Approximation and Range-Separated Hybrid Functionals

Extending GPU-Accelerated Gaussian Integrals in the TeraChem Software Package to f Type Orbitals: Implementation and Applications

Accelerated molecular dynamics force evaluation on graphics processing units for thermal conductivity calculations.

TTDFT: A GPU accelerated Tucker tensor DFT code for large-scale Kohn-Sham DFT calculations

Accelerating Coupled-Cluster Calculations with GPUs: An Implementation of the Density-Fitted CCSD(T) Approach for Heterogeneous Computing Architectures Using OpenMP Directives

A Parallel Implementation of the Analytic Nuclear Gradient for Time-Dependent Density Functional Theory Within the Tamm–Dancoff Approximation

On the Efficient Evaluation of the Exchange Correlation Potential on Graphics Processing Unit Clusters

Efficient Implementation of the Barnes-Hut Octree Algorithm for Monte Carlo Simulations of Charged Systems

Distributed Memory, GPU Accelerated Fock Construction for Hybrid, Gaussian Basis Density Functional Theory

Sparse tensor based nuclear gradients for periodic Hartree-Fock and low-scaling correlated wave function methods in the CP2K software package: A massively parallel and GPU accelerated implementation

An Efficient Multi-GPU Implementation for Linear-Response Time-Dependent Density Functional Theory

Tensor hypercontraction for fully self-consistent imaginary-time GF2 and GWSOX methods: theory, implementation, and role of the Green's function second-order exchange for intermolecular interactions

Efficient numerical solver for first-principles transport calculation based on real-space finite-difference method

Fast real-time time-dependent hybrid functional calculations with the parallel transport gauge and the adaptively compressed exchange formulation

Tucker tensor approach for accelerating exchange computations in a real-space finite-element discretization of generalized Kohn-Sham density functional theory

Efficient Tree Tensor Network States (TTNS) for Quantum Chemistry: Generalizations of the Density Matrix Renormalization Group Algorithm

Accelerating Atomic Orbital-based Electronic Structure Calculation via Pole Expansion and Selected Inversion

Iterative subspace algorithms for finite-temperature solution of Dyson equation

Achieving Performance Portability in Gaussian Basis Set Density Functional Theory on Accelerator Based Architectures in NWChemEx