Abstract:The International Journal of High Performance Computing Applications, Ahead of Print. As HPC system architectures and the applications running on them continue to evolve, the MPI standard itself must evolve. The trend in current and future HPC systems toward powerful nodes with multiple CPU cores and multiple GPU accelerators makes efficient support for hybrid programming critical for applications to achieve high performance. However, the support for hybrid programming in the MPI standard has not kept up with recent trends. The MPICH implementation of MPI provides a platform for implementing and experimenting with new proposals and extensions to fill this gap and to gain valuable experience and feedback before the MPI Forum can consider them for standardization. In this work, we detail six extensions implemented in MPICH to increase MPI interoperability with other runtimes, with a specific focus on heterogeneous architectures. First, the extension to MPI generalized requests lets applications integrate asynchronous tasks into MPI's progress engine. Second, the iovec extension to datatypes lets applications use MPI datatypes as a general-purpose data layout API beyond just MPI communications. Third, a new MPI object, MPIX_Stream, can be used by applications to identify execution contexts beyond MPI processes, including threads and GPU streams. MPIX stream communicators can be created to make existing MPI functions thread-aware and GPU-aware, thus providing applications with explicit ways to achieve higher performance. Fourth, MPIX Streams are extended to support the enqueue semantics for offloading MPI communications onto a GPU stream context. Fifth, thread communicators allow MPI communicators to be constructed with individual threads, thus providing a new level of interoperability between MPI and on-node runtimes such as OpenMP. Lastly, we present an extension to invoke MPI progress, which lets users spawn progress threads with fine-grained control to adapt the communication performance to their application designs. We describe the design and implementation of these extensions, provide usage examples, and highlight their expected benefits with performance results.

Research and Implementation of MPI on CELL Broad Engine

Analysis and MPI Implementation of LQCD Dslash on Sunway TaihuLight*

Accelerating Apache Hive with MPI for Data Warehouse Systems

Multi-GPU Hybrid Programming Accelerated Three-Dimensional Phase-Field Model in Binary Alloy

Implementation and Evaluation of MPI+OpenMP Programming Model on Dawning3000.

Improving the Performance of MPI Applications over Computational Grid.

Implementation and optimization of OpenMP task parallelism on heterogeneous multi-core architecture

Parallel approach of three-dimensional phase-field model of binary alloy in MPI+OpenMP environment

Designing and prototyping extensions to the Message Passing Interface in MPICH

Efficient Nonserial Polyadic Dynamic Programming on the Cell Processor

MAPE: A MIMD Architecture Performance Evaluating Simulation Environment

Source Code Migration of CUDA Program to Cell Platform

An MPI Implementation for Embeded Real Time Operation System

Toward exploitation of cell multi-processor array in time-consuming applications by using CNN model

An Implementation of Message-Passing Interface over VxWorks for Real-Time Embedded Multi-Core Systems

Cell-list Based Molecular Dynamics on Many-Core Processors: a Case Study on Sunway TaihuLight Supercomputer.

MPI Progress For All

ON INTER-PROCESSOR COMMUNICATION MECHANISM BASED ON ARM11 MPCORE

Using the High-Level Based Program Interface to Facilitate the Large Scale Scientific Computing

Data parallelization of Kd-tree ray tracing on the cell broadband engine

A Process Fusion Approach for MPI Performance Enhancement on Multi-core Systems.