Abstract:Heterogeneous parallel computing applications often process large data sets that require multiple GPUs to jointly meet their needs for physical memory capacity and compute throughput. However, the lack of high-level abstractions in previous heterogeneous parallel programming models force programmers to resort to multiple code versions, complex data copy steps and synchronization schemes when exchanging data between multiple GPU devices, which results in high software development cost, poor maintainability, and even poor performance. This paper describes the HPE runtime system, and the associated architecture support, which enables a simple, efficient programming interface for exchanging data between multiple GPUs through either interconnects or cross-node network interfaces. The runtime and architecture support presented in this paper can also be used to support other types of accelerators. We show that the simplified programming interface reduces programming complexity. The research presented in this paper started in 2009. It has been implemented and tested extensively in several generations of HPE runtime systems as well as adopted into the NVIDIA GPU hardware and drivers for CUDA 4.0 and beyond since 2011. The availability of real hardware that support key HPE features gives rise to a rare opportunity for studying the effectiveness of the hardware support by running important benchmarks on real runtime and hardware. Experimental results show that in a exemplar heterogeneous system, peer DMA and double-buffering, pinned buffers, and software techniques can improve the inter-accelerator data communication bandwidth by 2×. They can also improve the execution speed by 1.6× for a 3D finite difference, 2.5× for 1D FFT, and 1.6× for merge sort, all measured on real hardware. The proposed architecture support enables the HPE runtime to transparently deploy these optimizations under simple portable user code, allowing system designers to freely employ devices of different capabilities. We further argue that simple interfaces such as HPE are needed for most applications to benefit from advanced hardware features in practice.

Mamba: Portable Array-based Abstractions for Heterogeneous High-Performance Systems

An approach to performance portability through generic programming

Hybrid programming-model strategies for GPU offloading of electronic structure calculation kernels

Runtime Support for Performance Portability on Heterogeneous Distributed Platforms

Performance Portability of Sparse Block Diagonal Matrix Multiple Vector Multiplications on GPUs

Matrix-free approaches for GPU acceleration of a high-order finite element hydrodynamics application using MFEM, Umpire, and RAJA

A Lightweight Approach to Performance Portability with targetDP

Runtime and Architecture Support for Efficient Data Exchange in Multi-Accelerator Applications

Implementing Performance Portability of High Performance Computing Programs in the New Golden Age of Chip Architecture

High level programming abstractions for leveraging hierarchical memories with micro-core architectures

Application of performance portability solutions for GPUs and many-core CPUs to track reconstruction kernels

Automatic BLAS Offloading on Unified Memory Architecture: A Study on NVIDIA Grace-Hopper

Exploring the Performance Benefit of Hybrid Memory System on HPC Environments

PARRAY

Online Application Guidance for Heterogeneous Memory Systems

POPA: Expressing High and Portable Performance Across Spatial and Vector Architectures for Tensor Computations

MAGMA: Enabling exascale performance with accelerated BLAS and LAPACK for diverse GPU architectures

Performance Portable Monte Carlo Neutron Transport in MCDC via Numba

Performance and Power Efficient Massive Parallel Computational Model for HPC Heterogeneous Exascale Systems

Ginkgo -- A Math Library designed for Platform Portability

Taking GPU Programming Models to Task for Performance Portability