Abstract:Message Passing Interface (MPI) is the de facto standard for parallel programming, and collective operations in MPI are widely utilized by numerous scientific applications. The efficiency of these collective operations greatly impacts the performance of parallel applications. With the increasing scale and heterogeneity of HPC systems, the network environment has become more complex. The network states vary widely and dynamically between node pairs, and this makes it more difficult to design efficient collective communication algorithms. In this paper, we propose a method to optimize collective operations by using real-time measured network states, specifically focusing on the binomial tree algorithm. Our approach employs a low-overhead method to measure the network states, and the binomial tree with small latency is constructed based on the measurement result. Additionally, we take into account the disparities between the two underlying MPI peer-to-peer communication protocols, eager and rendezvous, and design tailored binomial tree construction algorithms for each protocol. We have implemented hierarchical MPI_Bcast, MPI_Reduce, MPI_Gather and MPI_Scatter, utilizing our network states-aware binomial tree algorithm at the inter-node level. The benchmark results demonstrate that our algorithm effectively enhances performance in small and medium message communication when compared to the default binomial tree algorithm in Open MPI. Specifically, for MPI_Bcast, we observe an average performance improvement of over 15.5% when the message size is less than 64KB. Similarly, for MPI_Reduce, there is an average performance improvement of over 12.1% when the message is below 2KB. In addition, there is an average performance improvement of over 10% for MPI_Gather when the message ranging from 64B to 512B. For MPI_Scatter, our algorithm achieved performance improvement only for certain message sizes.

Customized Network-on-Chip Oriented to MPI Collective Operations

Customized Network-on-Chip for Message Reduction.

Network states-aware collective communication optimization

Statistical time division multiplexing based local system architecture for multi-cluster NoC

A Power-Efficient Network-On-Chip for Multi-Core Stream Processors

Self-optimizing Two-layer Network-on-Chip Based on Dominant Network-Flow Adaption

High Performance Network-on-Chips (NoCs) Design: Performance Modeling, Routing Algorithm and Architecture Optimization

A hybird hierarchical architecture for 3D multi-cluster NoC

A tree-recursive partitioned multicast mechanism for NoC-based deep neural network accelerator

Application Mapping and Scheduling for Network-on-Chip-Based Multiprocessor System-on-Chip With Fine-Grain Communication Optimization

A Multi-Phase Based Multi-Application Mapping Approach for Many-Core Networks-on-Chip

Hybrid Parallel Programming Model for Hierarchical NoC

An Efficient Task Mapping Algorithm with Power-Aware Optimization for Network on Chip

Run-time Accelerate Channel for Communication-Aware Network-on-Chip

A Power-Aware Adaptive Routing Scheme for Network on a Chip

A Tightly Coupled Network-on-Chip Router Architecture

An Efficient and Low-Overhead Chip-to-Chip Interconnect Protocol Design for NOC

NUMA-aware shared-memory collective communication for MPI

Cost-Effective Power-Aware Core Testing in NoCs Based on a New Unicast-Based Multicast Scheme

Cluster Generation and Network Component Insertion for Topology Synthesis of Application-Specific Network-On-Chips