Abstract:Message Passing Interface (MPI) is the de facto standard for parallel programming, and collective operations in MPI are widely utilized by numerous scientific applications. The efficiency of these collective operations greatly impacts the performance of parallel applications. With the increasing scale and heterogeneity of HPC systems, the network environment has become more complex. The network states vary widely and dynamically between node pairs, and this makes it more difficult to design efficient collective communication algorithms. In this paper, we propose a method to optimize collective operations by using real-time measured network states, specifically focusing on the binomial tree algorithm. Our approach employs a low-overhead method to measure the network states, and the binomial tree with small latency is constructed based on the measurement result. Additionally, we take into account the disparities between the two underlying MPI peer-to-peer communication protocols, eager and rendezvous, and design tailored binomial tree construction algorithms for each protocol. We have implemented hierarchical MPI_Bcast, MPI_Reduce, MPI_Gather and MPI_Scatter, utilizing our network states-aware binomial tree algorithm at the inter-node level. The benchmark results demonstrate that our algorithm effectively enhances performance in small and medium message communication when compared to the default binomial tree algorithm in Open MPI. Specifically, for MPI_Bcast, we observe an average performance improvement of over 15.5% when the message size is less than 64KB. Similarly, for MPI_Reduce, there is an average performance improvement of over 12.1% when the message is below 2KB. In addition, there is an average performance improvement of over 10% for MPI_Gather when the message ranging from 64B to 512B. For MPI_Scatter, our algorithm achieved performance improvement only for certain message sizes.

Fibonacci Series-Based Multicast Algorithm

A novel distributed multicast routing algorithm based on ant colony algorithm

A Fast Routing Algorithm for Group Multicast with Bandwidth Reservation

Distributed Approximation Algorithms for Longest-Lived Multicast in WANETs with Directional Antennas

A Real-Time Multicast Routing Algorithm Based on the Genetic Algorithm

Network states-aware collective communication optimization

A Distributed Algorithm to Construct Multicast Trees in Wireless Multi-Hop Networks

A Novel Cooperative Multicast Scheme Based on Fountain Code

Multicast Tree Algorithm in ATM Network Environment

A New Reliable Multicast Algorithm on Ethernet

Lifetime Approximation Schemes Allow Multicast Algorithm with Linear Message Complexity in Wireless Sensor Networks

MULTICASTING ADDRESS COALESCING ALGORITHMS BASED ON BROADCAST BANYAN NETWORK

High-performance Broadcasting Algorithms on Cluster

Multiple Multicast for a Half-Duplex Butterfly Network: a Deterministic Approach

A Message Complexity Oriented Design Of Distributed Algorithm For Long-Lived Multicasting In Wireless Sensor Networks

A Genetic Algorithm-Based Multicast Tree for Routing in Pub/sub System.

A Novel Fast Multicast Algorithm with Enhanced Survivability in Wdm Optical Networks

New mechanism of two-layered multi-center multicast with ordering and its implementation

Tree-based Multicast Algorithm for 2D-Mesh Multiprocessor System

Fanout Splitting Multicast Scheduling in High Performance Router

Multicast Capacity Gain and Routing Algorithm in Static Network with Network Coding