Abstract:In this article, we propose a deep reinforcement learning (DRL)-based mobility load balancing (MLB) algorithm along with a two-layer architecture to solve the large-scale load balancing problem for ultradense networks (UDNs). Our contribution is threefold. First, this article proposes a two-layer architecture to solve the large-scale load balancing problem in a self-organized manner. The proposed architecture can alleviate the global traffic variations by dynamically grouping small cells into self-organized clusters according to their historical loads, and further adapt to local traffic variations through intracluster load balancing afterwards. Second, for the intracluster load balancing, this article proposes an off-policy DRL-based MLB algorithm to autonomously learn the optimal MLB policy under an asynchronous parallel learning framework, without any prior knowledge assumed over the underlying UDN environments. Moreover, the algorithm enables joint exploration with multiple behavior policies, such that the traditional MLB methods can be used to guide the learning process thereby improving the learning efficiency and stability. Third, this article proposes an offline-evaluation-based safeguard mechanism to ensure that the online system can always operate with the optimal and well-trained MLB policy, which not only stabilizes the online performance but also enables the exploration beyond current policies to make full use of machine learning in a safe way. Empirical results verify that the proposed framework outperforms the existing MLB methods in general UDN environments featured with irregular network topologies, coupled interferences, and random user movements, in terms of the load balancing performance.

BULB: Lightweight and Automated Load Balancing for Fast Datacenter Networks

DeepRLB: A deep reinforcement learning‐based load balancing in data center networks

Libra: A Stateful Layer-4 Load Balancer with Fair Load Distribution.

DRL-TAL: Deep Reinforcement Learning-Based Traffic-Aware Load Balancing in Data Center Networks

UFLB: A Unified Framework for Modeling and Analyzing Load Balancing Methods in DCNs

Load Balancing for Ultradense Networks: A Deep Reinforcement Learning-Based Approach.

Load balancing for heterogeneous traffic in datacenter networks

BOBBLE: A Mixed Routing-Granularity Distributed Load Balancing for Data Center Networks.

Load Balancing Fat-Tree on Long-Lived Flows： Avoiding Congestion in a Data Center Network

Towards Safe Load Balancing based on Control Barrier Functions and Deep Reinforcement Learning

PDLB: Path Diversity-aware Load Balancing with Adaptive Granularity in Data Center Networks

RSLB: Robust and Scalable Load Balancing in Software-Defined Data Center Networks.

Intelligent load balancing in data center software‐defined networks

An SDN-based Hybrid Strategy for Load Balancing in Data Center Networks

Safe Load Balancing in Software-Defined-Networking

BurstBalancer: Do Less, Better Balance for Large-scale Data Center Traffic

Distancer: A Host-Based Distributed Adaptive Load Balancer for Datacenter Traffic.

Reinforced Workload Distribution Fairness

PLB: Adaptive Partial Congestion-aware Load Balancing for Datacenter Networks

MEET: rack-level pooling based load balancing in datacenter networks