Abstract:Message Passing Interface (MPI) is the de facto standard for parallel programming, and collective operations in MPI are widely utilized by numerous scientific applications. The efficiency of these collective operations greatly impacts the performance of parallel applications. With the increasing scale and heterogeneity of HPC systems, the network environment has become more complex. The network states vary widely and dynamically between node pairs, and this makes it more difficult to design efficient collective communication algorithms. In this paper, we propose a method to optimize collective operations by using real-time measured network states, specifically focusing on the binomial tree algorithm. Our approach employs a low-overhead method to measure the network states, and the binomial tree with small latency is constructed based on the measurement result. Additionally, we take into account the disparities between the two underlying MPI peer-to-peer communication protocols, eager and rendezvous, and design tailored binomial tree construction algorithms for each protocol. We have implemented hierarchical MPI_Bcast, MPI_Reduce, MPI_Gather and MPI_Scatter, utilizing our network states-aware binomial tree algorithm at the inter-node level. The benchmark results demonstrate that our algorithm effectively enhances performance in small and medium message communication when compared to the default binomial tree algorithm in Open MPI. Specifically, for MPI_Bcast, we observe an average performance improvement of over 15.5% when the message size is less than 64KB. Similarly, for MPI_Reduce, there is an average performance improvement of over 12.1% when the message is below 2KB. In addition, there is an average performance improvement of over 10% for MPI_Gather when the message ranging from 64B to 512B. For MPI_Scatter, our algorithm achieved performance improvement only for certain message sizes.

A Precomputation Method Based on Memory Utilization in Intensive Communication Environments

Delay-Optimal Computation Offloading for Computation-Constrained Mobile Edge Networks.

Optimizing Peak Age of Information in MEC Systems: Computing Preemption and Non-preemption

A Staged Memory Resource Management Method for CMP Systems

Closed-Form Delay-Optimal Computation Offloading in Mobile Edge Computing Systems

MURS: Mitigating Memory Pressure in Service-oriented Data Processing Systems

A$^3$PIM: An Automated, Analytic and Accurate Processing-in-Memory Offloader

Near Data Computation for Message-Passing Chip-Multiprocessors.

A Multicore Periodical Preemption Virtual Machine Scheduling Scheme to Improve the Performance of Computational Tasks

An Improved Design for UMP (ubiquitous Multi-processor) System

Combining PVT Simulation and Queuing Model for System-Level Performance Analysis of Main Memory System

Performance Evaluation to Optimize the UMP System Focusing on Network Transmission Speed

Combine thread with memory scheduling for maximizing performance in multi-core systems

Network states-aware collective communication optimization

A^3PIM: an Automated, Analytic and Accurate Processing-in-Memory Offloader

Analysis on Computation-Intensive Status Update in Mobile Edge Computing

EMBA: Efficient Memory Bandwidth Allocation to Improve Performance on Intel Commodity Processor

Minimizing Stack Memory for Partitioned Mixed-criticality Scheduling on Multiprocessor Platforms

In-Place Irregular Computation for Message-Passing Chip-Multiprocessors.

PUMA: from Simultaneous to Parallel for Shared Memory System in Multi-core

Efficient Loop Scheduling for Chip Multiprocessors with Non-Volatile Main Memory