Abstract:Message Passing Interface (MPI) is the de facto standard for parallel programming, and collective operations in MPI are widely utilized by numerous scientific applications. The efficiency of these collective operations greatly impacts the performance of parallel applications. With the increasing scale and heterogeneity of HPC systems, the network environment has become more complex. The network states vary widely and dynamically between node pairs, and this makes it more difficult to design efficient collective communication algorithms. In this paper, we propose a method to optimize collective operations by using real-time measured network states, specifically focusing on the binomial tree algorithm. Our approach employs a low-overhead method to measure the network states, and the binomial tree with small latency is constructed based on the measurement result. Additionally, we take into account the disparities between the two underlying MPI peer-to-peer communication protocols, eager and rendezvous, and design tailored binomial tree construction algorithms for each protocol. We have implemented hierarchical MPI_Bcast, MPI_Reduce, MPI_Gather and MPI_Scatter, utilizing our network states-aware binomial tree algorithm at the inter-node level. The benchmark results demonstrate that our algorithm effectively enhances performance in small and medium message communication when compared to the default binomial tree algorithm in Open MPI. Specifically, for MPI_Bcast, we observe an average performance improvement of over 15.5% when the message size is less than 64KB. Similarly, for MPI_Reduce, there is an average performance improvement of over 12.1% when the message is below 2KB. In addition, there is an average performance improvement of over 10% for MPI_Gather when the message ranging from 64B to 512B. For MPI_Scatter, our algorithm achieved performance improvement only for certain message sizes.

Optimal collective communication algorithms in grid computing system

A Deadline And Budget Constrained Cost-Time Optimization Algorithm For Scheduling Dependent Tasks In Grid Computing

A Public Grid Computing Framework Based on a Hierarchical Combination of Middleware

Delocalization and spin-wave dynamics in ferromagnetic chains with long-range correlated random exchange

Effective method of collective communication for message passing on cluster

Latency-Balanced Optimization of MPI Collective Communication across Multi-clusters

Multicast communication in grid computing network for real-time streaming media data

Improving the Performance of MPI Applications over Computational Grid.

Research of the Grid Computing System Applied in Optical Simulation

Applying Computational Grid Technology to Power System

Study On The Performance Of Grid Computing In Power System Distributed Computing

Network states-aware collective communication optimization

Collective Communication Optimization for Solving Linear Algebraic Equations

A Distributed Cooperative Technology for Spatial Grid Computing

Optimizing MPI Collectives on Shared Memory Multi-Cores

A Survey of Methods for Collective Communication Optimization and Tuning

Optimizing Irregular Communication with Neighborhood Collectives and Locality-Aware Parallelism

Network Performance-aware Communication Model for Clustered Wide Area Systems

The application of an improved cultural algorithm in grid computing

A Novel Routing and Wavelength Assignment Algorithm for Multicast in Optical Grid Networks

Design and Implementation of a Universal Grid Computing Platform Applicable to Optimal Operation of Power Grid