Abstract:The International Journal of High Performance Computing Applications, Ahead of Print. As HPC system architectures and the applications running on them continue to evolve, the MPI standard itself must evolve. The trend in current and future HPC systems toward powerful nodes with multiple CPU cores and multiple GPU accelerators makes efficient support for hybrid programming critical for applications to achieve high performance. However, the support for hybrid programming in the MPI standard has not kept up with recent trends. The MPICH implementation of MPI provides a platform for implementing and experimenting with new proposals and extensions to fill this gap and to gain valuable experience and feedback before the MPI Forum can consider them for standardization. In this work, we detail six extensions implemented in MPICH to increase MPI interoperability with other runtimes, with a specific focus on heterogeneous architectures. First, the extension to MPI generalized requests lets applications integrate asynchronous tasks into MPI's progress engine. Second, the iovec extension to datatypes lets applications use MPI datatypes as a general-purpose data layout API beyond just MPI communications. Third, a new MPI object, MPIX_Stream, can be used by applications to identify execution contexts beyond MPI processes, including threads and GPU streams. MPIX stream communicators can be created to make existing MPI functions thread-aware and GPU-aware, thus providing applications with explicit ways to achieve higher performance. Fourth, MPIX Streams are extended to support the enqueue semantics for offloading MPI communications onto a GPU stream context. Fifth, thread communicators allow MPI communicators to be constructed with individual threads, thus providing a new level of interoperability between MPI and on-node runtimes such as OpenMP. Lastly, we present an extension to invoke MPI progress, which lets users spawn progress threads with fine-grained control to adapt the communication performance to their application designs. We describe the design and implementation of these extensions, provide usage examples, and highlight their expected benefits with performance results.

Asynchronous progress design for a MPI-based PGAS one-sided communication system

MPI Progress For All

Towards a Scalable and Efficient PGAS-based Distributed OpenMP

Enabling Highly-Scalable Remote Memory Access Programming with MPI-3 One Sided

Characterizing the Performance of Node-Aware Strategies for Irregular Point-to-Point Communication on Heterogeneous Architectures

Optimizing Irregular Communication with Neighborhood Collectives and Locality-Aware Parallelism

Designing and prototyping extensions to the Message Passing Interface in MPICH

Asynchronous Complex Analytics in a Distributed Dataflow Architecture

Modeling Data Movement Performance on Heterogeneous Architectures

Magas: matrix-based asynchronous graph analytics on shared memory systems

An Asynchronous Algorithm for Optimizing the Communication Performance

Driving asynchronous distributed tasks with events

Accelerating GPU Message Communication for Autonomous Navigation Systems.

LLVM-based communication optimizations for PGAS programs

Improving Communication Patterns in Polyhedral Process Networks

Structures and Techniques for Streaming Dynamic Graph Processing on Decentralized Message-Driven Systems

A More Scalable Sparse Dynamic Data Exchange

An Efficient Shared Memory Based Virtual Communication System for Embedded SMP Cluster

Process Mapping for MPI Collective Communications.

AIR: A Light-Weight Yet High-Performance Dataflow Engine based on Asynchronous Iterative Routing

Efficient Distributed Memory Management with RDMA and Caching