Abstract:Heterogeneous nodes composed of a multicore CPU and accelerators are today's norm in high-performance computing (HPC) platforms due to their superior performance and energy efficiency. Tools such as OpenCL and hybrid combinations such as OpenMP plus OpenACC are used for developing portable parallel programs for such nodes. However, these tools have some drawbacks, including a lack of compiler support for nested parallelism, performance portability, automatic heterogeneous workload distribution, user-friendly thread placement, and processor affinity essential to the portable performance of hybrid programs executing on such nodes. In this paper, we propose OpenH, a novel programming model and library API for developing portable parallel programs on heterogeneous hybrid servers composed of a multicore CPU and one or more different types of accelerators. OpenH integrates Pthreads, OpenMP, and OpenACC seamlessly to facilitate the development of hybrid parallel programs. An OpenH hybrid parallel program starts as a single main thread, creating a group of Pthreads called hosting Pthreads. A hosting Pthread then leads the execution of a software component of the program, either an OpenMP multithreaded component running on the CPU cores or an OpenACC (or OpenMP) component running on one of the accelerators of the server. The OpenH library provides API functions that allow programmers to get the configuration of the executing environment and bind the hosting Pthreads (and hence the execution of components) of the program to the CPU cores of the hybrid server to get the best performance. We illustrate the OpenH programming model and library API using two hybrid parallel applications based on matrix multiplication and 2D fast Fourier transform for the most general case of a hybrid hyperthreaded server comprising computing devices. Finally, we demonstrate the practical performance and energy consumption of OpenH for the hybrid parallel matrix multiplication application on a server comprising an Intel Icelake multicore CPU and two Nvidia A40 GPUs.

Large-scale shape changes in proteins and macromolecular complexes.

Runtime Support for Performance Portability on Heterogeneous Distributed Platforms

HPVM-HDC: A Heterogeneous Programming System for Accelerating Hyperdimensional Computing

OpenH: A Novel Programming Model and API for Developing Portable Parallel Programs on Heterogeneous Hybrid Servers

Efficient Resource Sharing Through GPU Virtualization on Accelerated High Performance Computing Systems

Portability and Scalability of OpenMP Offloading on State-of-the-art Accelerators

Simultaneous and Heterogenous Multithreading: Exploiting Simultaneous and Heterogeneous Parallelism in Accelerator-Rich Architectures

Application of performance portability solutions for GPUs and many-core CPUs to track reconstruction kernels

Efficient GPU Spatial-Temporal Multitasking

Dwarfs on Accelerators: Enhancing OpenCL Benchmarking for Heterogeneous Computing Architectures

Understanding Data Movement in Tightly Coupled Heterogeneous Systems: A Case Study with the Grace Hopper Superchip

Performance and Power Efficient Massive Parallel Computational Model for HPC Heterogeneous Exascale Systems

Automatic execution of single-GPU computations across multiple GPUs

Exploiting Memory Access Patterns to Improve Memory Performance in Data-Parallel Architectures

AN PARALLEL AND DISTRIBUTED PROGRAMMING SOLUTION BASED ON HETEROGENEOUS GPU CLUSTER

StarPU: a unified platform for task scheduling on heterogeneous multicore architectures

Application Experiences on a GPU-Accelerated Arm-based HPC Testbed

HPX -- An open source C++ Standard Library for Parallelism and Concurrency

Optimizing Offload Performance in Heterogeneous MPSoCs

Heterogeneous architectures for computational intensive applications: A cost-effectiveness analysis

Implementing Performance Portability of High Performance Computing Programs in the New Golden Age of Chip Architecture