Abstract:The International Journal of High Performance Computing Applications, Ahead of Print. As HPC system architectures and the applications running on them continue to evolve, the MPI standard itself must evolve. The trend in current and future HPC systems toward powerful nodes with multiple CPU cores and multiple GPU accelerators makes efficient support for hybrid programming critical for applications to achieve high performance. However, the support for hybrid programming in the MPI standard has not kept up with recent trends. The MPICH implementation of MPI provides a platform for implementing and experimenting with new proposals and extensions to fill this gap and to gain valuable experience and feedback before the MPI Forum can consider them for standardization. In this work, we detail six extensions implemented in MPICH to increase MPI interoperability with other runtimes, with a specific focus on heterogeneous architectures. First, the extension to MPI generalized requests lets applications integrate asynchronous tasks into MPI's progress engine. Second, the iovec extension to datatypes lets applications use MPI datatypes as a general-purpose data layout API beyond just MPI communications. Third, a new MPI object, MPIX_Stream, can be used by applications to identify execution contexts beyond MPI processes, including threads and GPU streams. MPIX stream communicators can be created to make existing MPI functions thread-aware and GPU-aware, thus providing applications with explicit ways to achieve higher performance. Fourth, MPIX Streams are extended to support the enqueue semantics for offloading MPI communications onto a GPU stream context. Fifth, thread communicators allow MPI communicators to be constructed with individual threads, thus providing a new level of interoperability between MPI and on-node runtimes such as OpenMP. Lastly, we present an extension to invoke MPI progress, which lets users spawn progress threads with fine-grained control to adapt the communication performance to their application designs. We describe the design and implementation of these extensions, provide usage examples, and highlight their expected benefits with performance results.

A Process Fusion Approach for MPI Performance Enhancement on Multi-core Systems.

Optimizing MPI Collectives on Shared Memory Multi-Cores

Performance Evaluation and Optimization of Inter-cores Communication for Heterogeneous Multi-core Processor Unit

Parallel approach of three-dimensional phase-field model of binary alloy in MPI+OpenMP environment

A Multi-Core Parallel Fusion Algorithm for Remote-Sensing Image

Performance Optimization of a CFD Application on Intel Multicore and Manycore Architectures

Designing and prototyping extensions to the Message Passing Interface in MPICH

MPI Progress For All

Faster and Scalable MPI Applications Launching

MPI+Threads: runtime contention and remedies

Improving the Performance of MPI Applications over Computational Grid.

Some New Approaches to MPI Implementations

Automatic Tuning of Sparse Matrix-Vector Multiplication on Multicore Clusters.

Process Mapping for MPI Collective Communications.

On the Parallelization Optimization Strategy for High Performance Computing Software

Optimization Strategies of Data Processing Algorithms under Multi-Core Architecture

Cache Coherence Method for Improving Multi-threaded Applications on Multicore Systems

An Approach to Enhance the Performance of Large‐scale Structural Analysis on CPU‐MIC Heterogeneous Clusters

An Efficient Protocol with Synchronization Accelerator for Multi-Processor Embedded Systems

Accelerating Data Movement on Future Chip Multi-Processors

MPIPP: an automatic profile-guided parallel process placement toolset for SMP clusters and multiclusters.