Abstract:Data center networks (DCNs) are facing challenging control problems such as flow scheduling, congestion control, load balancing, and bandwidth allocation when dynamically handling heterogeneous mice and elephant flows. This paper focuses on the load balancing in DCNs as an online decision-making problem with high complexity (NP-hard problem). Traditional load balancing approaches are usually the heuristic algorithms relying on the operator's viewpoint and a short-term knowledge of the traffic conditions and network environment. This paper proposes DeepRLB, a deep reinforcement learning (DRL)-based load balancing approach for software-defined networking-based DCNs which exploits the deep deterministic policy gradient (DDPG) algorithm to adaptively learn the link-weight values by observing the traffic flow characteristics. The actor and critic neural networks in the DDPG model have been designed based on two learning models: fully connected (DeepRLB-Full) and convolutional (DeepRLB-Conv). The output of the model is further used by the controller to determine the forwarding paths for traffic flows accordingly. We evaluated the performance of DeepRLB by comparing it with ECMP as well as a per-packet load balancing algorithm in three different data center topologies under 30 heterogeneous traffic demand matrices. The obtained results showed that the DeepRLB algorithm outperforms the ECMP with respect to studied load balancing metrics considerably (DeepRLB-Conv achieved the best performance). As a sample, DeepRLB-Full and DeepRLB-Conv were able to reach 19.14% and 24.46% reduction in the total number of over-utilized and under-utilized network links compared to ECMP, respectively.

DeepRLB: A deep reinforcement learning‐based load balancing in data center networks