Abstract:With the widespread use of distributed machine learning (DML), many IT companies have established networks dedicated to DML. Different communication architectures of DML have different traffic patterns and different requirements on network performance, which is closely related to network topology. However, traditional network topologies usually pursue general goals and are agnostic to the special communication pattern of the applications. The mismatch between network topology and the applications will directly affect the training performance. Although some studies have analyzed the effect of topology on training performance, the topologies and communication architectures involved are not comprehensive, and it is still not known which topology is appropriate for which communication architecture. This survey investigates typical topologies and analyzes whether they meet the requirements of three commonly used communication architectures (i.e., Parameter Server (PS), Tree and Ring architectures) of DML. Specifically, the topology requirements of each communication architecture and two common topology requirements (i.e., high scalability and fault tolerance) for DML are studied firstly. Next, whether these topologies meet the topology requirements is analyzed. Then, this paper discusses potential technologies and approaches to construct the appropriate scheme for each topology requirement, and then presents DMLNet, a novel network topology that suits the three communication architectures. Finally, several potential directions for future research are outlined.

Survey on Application of Communication Network Topology Simulation Technology

OMNeT++ Based Simulation for Topology Control in Wireless Sensor Network: A Case Study

Study on Different Topology Manipulation Algorithms in Wireless Sensor Network

On the Simulation of Networked Sensor Systems.

Nettopo: Beyond Simulator And Visualizer For Wireless Sensor Networks

Modeling Analysis of Network Topology Based on Distribution Network Automation

Research and Development of Network Traffic Simulation

Behaviours of Networks with Different Topologies and Protocols

Design of Complex Communication Network Based on Simulation

Network Simulators for Satellite-Terrestrial Integrated Networks: A Survey

Simulation Model of Topology Generation in Hierarchical Distributed Ad Hoc Networks

A Research on Integrated Space-Ground Information Network Simulation Platform Based on SDN

Survey on the Networks-on-Chip Interconnection Topologies

A Survey on the Internet Complexity

Study of Various Network Topologies Using Graph Theory

The Survey of Complex Dynamical Networks Models

Network Simulation Based on OPNTE and Application

Simulation and Improved Methodology of Eurocom Communication Protocol

A Study And Analysis on Computer Network Topology For Data Communication

Topologies in distributed machine learning: Comprehensive survey, recommendations and future directions

Research on Simulation Technology for Space-ground Integrated Network