Abstract:Networking has become a well-known performance bottleneck for distributed machine learning (DML). Although lots of works have focused on accelerating the communication process of DML, they ignore the impact of the physical network on the DML performance. Concurrently, optical circuit switches (OCSes) are increasingly applied in data centers and clusters, which can fundamentally improve DML performance. It's worth noting that the non-negligible OCS reconfiguration delay makes OCS scheduling algorithms have a great impact on the upper application performance. However, existing OCS scheduling solutions are not suitable for DML jobs due to the iterative nature of DML jobs and their interleaving characteristics of communication and computation stages. Therefore, in this paper, we study the online multi-job scheduling for DML in OCS networks. Firstly, we propose heaviest-load-first (HLF), a heuristic algorithm for intra-job scheduling, which is based on the fact that the completion time of flows on the heaviest load port has a significant impact on the job completion time. Furthermore, we present Shortest Weighted Remaining Time First (SWRTF) algorithm for inter-job scheduling. In SWRTF, an available DML job is scheduled when the served job moves from communication stage to the computation stage, which significantly improves the circuit utilization. Based on large-scale simulations, we demonstrate HLF can significantly reduce the iteration communication time by up to 64.97% compared to the state-of-the-art circuit scheduler Sunflow. Besides, SWRTF can save up to 42.9%, 54.2%, 27.2% of Weighted-Job-Completion-Time (WJCT) compared to Shortest-Job-First, Baraat and Weighted-First inter-job scheduling algorithms, respectively.

Time-Division Based Scheduling Scheme for Hybrid Optical/Electrical Data Center Network

Hybrid Packet/Fine-Grained-Optical Switching Architectures for Future Large-Scale Data Center Networks

Effective *-Flow Schedule for Optical Circuit Switching Based Data Center Networks: A Comprehensive Survey

Scheduling Virtual Network Reconfigurations in Parallel in Hybrid Optical/Electrical Datacenter Networks

Scalable Data Center Network with distributed placement of component sets in optical networks

Joint Scheduling and Routing for QoS Guaranteed Packet Transmission in Energy Efficient Reconfigurable WDM Mesh Networks

Time-Spectrum Consecutiveness Based Scheduling With Advance Reservation in Elastic Optical Networks

Fine-Grained All-Optical Switching Based on Optical Time Slice Switching for Hybrid Packet-Ocs Intra-Data Center Networks

Machine Learning Assisted Optical Network Resource Scheduling in Data Center Networks.

On the Performance of a Scalable Optical Switching Architecture for Flat Intercluster Data Center Network with Centralized Control

Agile and fine programmable optical interconnection provisioning in data center network

On Establishing and Task Scheduling of Data-Oriented Vnf-Scs in an Optical DCI

Online job scheduling for distributed machine learning in optical circuit switch networks

Scheduling Techniques for Hybrid Circuit/packet Networks

Data-Oriented Task Scheduling in Fixed- and Flexible-Grid Multilayer Inter-DC Optical Networks: A Comparison Study

Omco: Online Multiple Coflow Scheduling In Optical Circuit Switch

Timeslot Switching-Based Optical Bypass in Data Center for Intrarack Elephant Flow with an Ultrafast DPDK-Enabled Timeslot Allocator

Enabling Low Latency at Large-Scale Data Center and High-Performance Computing Interconnect Networks Using Fine-Grained All-Optical Switching Technology

Energy-efficient Scaling of Active Electrical/Optical Switches in Hybrid Packet/Circuit Switched Data Center Networks

A New Optoelectronic Hybrid Network Based on Scheduling Optimization of Optical Links

A Novel Distributed Scheduling Approach Powered by Central Databases in Optical Burst Networks