Abstract:The International Journal of High Performance Computing Applications, Ahead of Print. As HPC system architectures and the applications running on them continue to evolve, the MPI standard itself must evolve. The trend in current and future HPC systems toward powerful nodes with multiple CPU cores and multiple GPU accelerators makes efficient support for hybrid programming critical for applications to achieve high performance. However, the support for hybrid programming in the MPI standard has not kept up with recent trends. The MPICH implementation of MPI provides a platform for implementing and experimenting with new proposals and extensions to fill this gap and to gain valuable experience and feedback before the MPI Forum can consider them for standardization. In this work, we detail six extensions implemented in MPICH to increase MPI interoperability with other runtimes, with a specific focus on heterogeneous architectures. First, the extension to MPI generalized requests lets applications integrate asynchronous tasks into MPI's progress engine. Second, the iovec extension to datatypes lets applications use MPI datatypes as a general-purpose data layout API beyond just MPI communications. Third, a new MPI object, MPIX_Stream, can be used by applications to identify execution contexts beyond MPI processes, including threads and GPU streams. MPIX stream communicators can be created to make existing MPI functions thread-aware and GPU-aware, thus providing applications with explicit ways to achieve higher performance. Fourth, MPIX Streams are extended to support the enqueue semantics for offloading MPI communications onto a GPU stream context. Fifth, thread communicators allow MPI communicators to be constructed with individual threads, thus providing a new level of interoperability between MPI and on-node runtimes such as OpenMP. Lastly, we present an extension to invoke MPI progress, which lets users spawn progress threads with fine-grained control to adapt the communication performance to their application designs. We describe the design and implementation of these extensions, provide usage examples, and highlight their expected benefits with performance results.

MPICH-G-DM: An Enhanced MPICH-G with Supporting Dynamic Job Migration

A Flexible Architecture for Job Management in A Grid Environment

A Public Grid Computing Framework Based on a Hierarchical Combination of Middleware

MPICH-G2: A Grid-Enabled Implementation of the Message Passing Interface

Application-adaptive Resource Scheduling in a Computational Grid

User-Oriented Job Management in a Grid Environment

DataMPI: Extending MPI to Hadoop-Like Big Data Computing

Transparent Migration of Multi-Threaded Applications on a Java Based Grid

Multi-discipline computations and collaborative visualization on the campus grid

A Task Migration Mechanism for MPI Applications

Gmdh Based on Genetic Algorithmand Mmt Policy for Energy Efficient Dynamic Consolidation of Virtual Machines

Applying Computational Grid Technology to Power System

A Cooperative Process-Management System Based on the Manufacturing Grid

Designing and prototyping extensions to the Message Passing Interface in MPICH

PIVOT: An adaptive information discovery framework for computational grids

Process Migration for the Network of Workstations

Improving the Performance of MPI Applications over Computational Grid.

Multi-grid job scheduling

ADOM: An Adaptive Objective Migration Strategy for Grid Platform

Using the High-Level Based Program Interface to Facilitate the Large Scale Scientific Computing

Distributed Grid Computing Framework and Application in Molecular Dynamics Simulation