Abstract:Heterogeneous nodes composed of a multicore CPU and accelerators are today's norm in high-performance computing (HPC) platforms due to their superior performance and energy efficiency. Tools such as OpenCL and hybrid combinations such as OpenMP plus OpenACC are used for developing portable parallel programs for such nodes. However, these tools have some drawbacks, including a lack of compiler support for nested parallelism, performance portability, automatic heterogeneous workload distribution, user-friendly thread placement, and processor affinity essential to the portable performance of hybrid programs executing on such nodes. In this paper, we propose OpenH, a novel programming model and library API for developing portable parallel programs on heterogeneous hybrid servers composed of a multicore CPU and one or more different types of accelerators. OpenH integrates Pthreads, OpenMP, and OpenACC seamlessly to facilitate the development of hybrid parallel programs. An OpenH hybrid parallel program starts as a single main thread, creating a group of Pthreads called hosting Pthreads. A hosting Pthread then leads the execution of a software component of the program, either an OpenMP multithreaded component running on the CPU cores or an OpenACC (or OpenMP) component running on one of the accelerators of the server. The OpenH library provides API functions that allow programmers to get the configuration of the executing environment and bind the hosting Pthreads (and hence the execution of components) of the program to the CPU cores of the hybrid server to get the best performance. We illustrate the OpenH programming model and library API using two hybrid parallel applications based on matrix multiplication and 2D fast Fourier transform for the most general case of a hybrid hyperthreaded server comprising computing devices. Finally, we demonstrate the practical performance and energy consumption of OpenH for the hybrid parallel matrix multiplication application on a server comprising an Intel Icelake multicore CPU and two Nvidia A40 GPUs.

Multithreading Compilation for HPF Program Based on SMP Cluster System.

Design, Implementation of the Parallel C Language Based on C/S Mode in Distributed Systems

A Compiling Framework Based on Uniform Partitioning Schemes for Parallel Loops

AN PARALLEL AND DISTRIBUTED PROGRAMMING SOLUTION BASED ON HETEROGENEOUS GPU CLUSTER

The Approach to Support the P&M Multithreaded Program Execution Model on Workstation Cluster

Parallel Computing of Shared Memory Multiprocessors Based on JOMP

HPX -- An open source C++ Standard Library for Parallelism and Concurrency

High-level parallel computing language

OpenH: A Novel Programming Model and API for Developing Portable Parallel Programs on Heterogeneous Hybrid Servers

Simultaneous and Heterogenous Multithreading: Exploiting Simultaneous and Heterogeneous Parallelism in Accelerator-Rich Architectures

HeteroPP: A directive‐based heterogeneous cooperative parallel programming framework

5 ExaFlop/s HPL-MxP Benchmark with Linear Scalability on the 40-Million-Core Sunway Supercomputer.

Task Scheduling of Parallel Programs to Optimize Communications for Cluster of SMPs

StarPU: a unified platform for task scheduling on heterogeneous multicore architectures

Horde： A Framework for Parallel Programming on Multi-Core Clusters

High-Performance PVM Based on Fast Message Passing

Design and implementation of self-adaptable parallel algorithms for scientific computing on highly heterogeneous HPC platforms

Reverse Compilation for Speculative Parallel Threading

Compile-Time Scheduling Of Multithread With Data Localities On Multiple Vector Processors

Optimization of Computation-Intensive Applications in cc-NUMA Architecture

Experiences Porting Distributed Applications to Asynchronous Tasks: A Multidimensional FFT Case-study