Machine-learning-aided cognitive reconfiguration for flexible-bandwidth HPC and data center networks [Invited]

Xiaoliang Chen,Roberto Proietti,Marjan Fariborz,Che-Yu Liu,S. J. Ben Yoo
DOI: https://doi.org/10.1364/jocn.412360
IF: 4.508
2021-01-20
Journal of Optical Communications and Networking
Abstract:This paper proposes a machine-learning (ML)-aided cognitive approach for effective bandwidth reconfiguration in optically interconnected datacenter/high-performance computing (HPC) systems. The proposed approach relies on a Hyper-X-like architecture augmented with flexible-bandwidth photonic interconnections at large scales using a hierarchical intra/inter-POD photonic switching layout. We first formulate the problem of the connectivity graph and routing scheme optimization as a mixed-integer linear programming model. A two-phase heuristic algorithm and a joint optimization approach are devised to solve the problem with low time complexity. Then, we propose an ML-based end-to-end performance estimator design to assist the network control plane with intelligent decision making for bandwidth reconfiguration. Numerical simulations using traffic distribution profiles extracted from HPC applications traces as well as random traffic matrices verify the accuracy performance of the ML design estimator ( <math display="inline"> <mrow> <mo> &lt; </mo> </mrow> <mn>9</mn> <mi mathvariant="normal"> % </mi> </math> error) and demonstrate up to <math display="inline"> <mn>5</mn> <mo> × </mo> </math> throughput gain from the proposed approach compared with the baseline Hyper-X network using fixed all-to-all intra/inter-portable data center interconnects.
computer science, information systems,telecommunications, hardware & architecture,optics
What problem does this paper attempt to address?