Abstract:This study proposes a mechanism to handle stragglers in data‐parallel ML training by offloading the control of the SSP synchronization to the programmable data plane using a memory‐efficient data structure. This work also introduces an API for dynamic adjustment of synchronization barriers. Tested by injecting stragglers in a realistic ML workload, on both an emulated environment and real hardware, the system improves job completion time and waiting time with limited impact on model accuracy. Achieving high‐performance aggregation is essential to scaling data‐parallel distributed machine learning (ML) training. Recent research in in‐network computing has shown that offloading the aggregation to the network data plane can accelerate the aggregation process compared to traditional server‐only approaches, reducing the propagation delay and consequently speeding up distributed training. However, the existing literature on in‐network aggregation does not provide ways to deal with slower workers (called stragglers). The presence of stragglers can negatively impact distributed training, increasing the time it takes to complete. In this paper, we present Serene, an in‐network aggregation system capable of circumventing the effects of stragglers. Serene coordinates the ML workers to cooperate with a programmable switch using a hybrid synchronization approach where approaches can be changed dynamically. The synchronization can change dynamically through a control plane API that translates high‐level code into switch rules. Serene switch employs an efficient data structure for managing synchronization and a hot‐swapping mechanism to consistently change from one synchronization strategy to another. We implemented and evaluated a prototype using BMv2 and a Proof‐of‐Concept in a Tofino ASIC. We ran experiments with realistic ML workloads, including a neural network trained for image classification. Our results show that Serene can speed up training by up to 40% in emulation scenarios by reducing drastically the cumulative waiting time compared to a synchronous baseline.

Homunculus: Auto-Generating Efficient Data-Plane ML Pipelines for Datacenter Networks

AI in 6G: Energy-Efficient Distributed Machine Learning for Multilayer Heterogeneous Networks

Machine-learning-aided cognitive reconfiguration for flexible-bandwidth HPC and data center networks [Invited]

HOMMO: A Hierarchical Flow Management Framework for Multi-Objective Data Center Networks.

A Data-Centric Optimization Framework for Machine Learning

Rethinking Data Center Networks: Machine Learning Enables Network Intelligence

Holmes: Towards Distributed Training Across Clusters with Heterogeneous NIC Environment

Octopus: A Heterogeneous In-network Computing Accelerator Enabling Deep Learning for network

MLOps: Automatic, Zero-Touch and Reusable Machine Learning Training and Serving Pipelines

No Worker Left (Too Far) Behind: Dynamic Hybrid Synchronization for In‐Network ML Aggregation

Towards autonomic orchestration of machine learning pipelines in future networks

Machine Learning to Control Network Powered by Computing Infrastructure

Machine Learning Empowered Intelligent Data Center Networking: A Survey

DiviML: A Module-based Heuristic for Mapping Neural Networks onto Heterogeneous Platforms

IIsy: Hybrid In-Network Classification Using Programmable Switches

Optimizing network slicing in 6G networks through a hybrid deep learning strategy

Automating In-Network Machine Learning

OptimusPrime: Unleash Dataplane Programmability Through a Transformable Architecture

Auto-Scaling Network Resources using Machine Learning to Improve QoS and Reduce Cost

Scalable Power Control/Beamforming in Heterogeneous Wireless Networks with Graph Neural Networks

A Scalable, High-Performance, and Fault-Tolerant Network Architecture for Distributed Machine Learning