Abstract:The International Journal of High Performance Computing Applications, Ahead of Print. As HPC system architectures and the applications running on them continue to evolve, the MPI standard itself must evolve. The trend in current and future HPC systems toward powerful nodes with multiple CPU cores and multiple GPU accelerators makes efficient support for hybrid programming critical for applications to achieve high performance. However, the support for hybrid programming in the MPI standard has not kept up with recent trends. The MPICH implementation of MPI provides a platform for implementing and experimenting with new proposals and extensions to fill this gap and to gain valuable experience and feedback before the MPI Forum can consider them for standardization. In this work, we detail six extensions implemented in MPICH to increase MPI interoperability with other runtimes, with a specific focus on heterogeneous architectures. First, the extension to MPI generalized requests lets applications integrate asynchronous tasks into MPI's progress engine. Second, the iovec extension to datatypes lets applications use MPI datatypes as a general-purpose data layout API beyond just MPI communications. Third, a new MPI object, MPIX_Stream, can be used by applications to identify execution contexts beyond MPI processes, including threads and GPU streams. MPIX stream communicators can be created to make existing MPI functions thread-aware and GPU-aware, thus providing applications with explicit ways to achieve higher performance. Fourth, MPIX Streams are extended to support the enqueue semantics for offloading MPI communications onto a GPU stream context. Fifth, thread communicators allow MPI communicators to be constructed with individual threads, thus providing a new level of interoperability between MPI and on-node runtimes such as OpenMP. Lastly, we present an extension to invoke MPI progress, which lets users spawn progress threads with fine-grained control to adapt the communication performance to their application designs. We describe the design and implementation of these extensions, provide usage examples, and highlight their expected benefits with performance results.

The parallelization of SPIDER on distributed-memory computers using MPI.

Parallel Computing of Shared Memory Multiprocessors Based on JOMP

Evaluation and Analysis of Distributed Graph-Parallel Processing Frameworks

fgSpMSpV: A Fine-grained Parallel SpMSpV Framework on HPC Platforms

Mpi-Based Parallelized Precorrected Fft Algorithm for Analyzing Scattering by Arbitrarily Shaped Three-Dimensional Objects - Abstract

Mspr: A Retained-Mode Based Multi-Screen Parallel Rendering System

Parallelism for cryo-EM 3D reconstruction on CPU-GPU heterogeneous system

Proven Distributed Memory Parallelization of Particle Methods

Parallelization of Bayesian Network Based SNPs Pattern Analysis and Performance Characterization on SMP/HT

PARALLEL PROGRAMMING OF 2(1/2)-DIMENSIONAL PIC UNDER DISTRIBUTED-MEMORY PARALLEL ENVIRONMENTS

Experience Of Parallelizing Cryo-Em 3d Reconstruction On A Cpu-Gpu Heterogeneous System

Increasing the Efficiency of Sparse Matrix-Matrix Multiplication with a 2.5D Algorithm and One-Sided MPI

On the Parallelization Optimization Strategy for High Performance Computing Software

A massive MPI parallel framework of smoothed particle hydrodynamics with optimized memory management for extreme mechanics problems

Building Near-Real-Time Processing Pipelines with the Spark-MPI Platform

Designing and prototyping extensions to the Message Passing Interface in MPICH

Parallel algorithms and implement of explicit integration method for structural dynamic analysis

Parallel Algorithms for Dynamics Analysis of Structures and Their Impementations

Towards a Scalable and Efficient PGAS-based Distributed OpenMP

Study on the parallel algorithm of crop development simulation

Exploring the Design Space of Distributed Parallel Sparse Matrix-Multiple Vector Multiplication