Abstract:The Open Radio Access Network (O-RAN) architecture empowers intelligent and automated optimization of the RAN through applications deployed on the RAN Intelligent Controller (RIC) platform, enabling capabilities beyond what is achievable with traditional RAN solutions. Within this paradigm, Traffic Steering (TS) emerges as a pivotal RIC application that focuses on optimizing cell-level mobility settings in near-real-time, aiming to significantly improve network spectral efficiency. In this paper, we design a novel TS algorithm based on a Cascade Reinforcement Learning (CaRL) framework. We propose state space factorization and policy decomposition to reduce the need for large models and well-labeled datasets. For each sub-state space, an RL sub-policy will be trained to learn an optimized mapping onto the action space. To apply CaRL on new network regions, we propose a knowledge transfer approach to initialize a new sub-policy based on knowledge learned by the trained policies. To evaluate CaRL, we build a data-driven and scalable RIC digital twin (DT) that is modeled using important real-world data, including network configuration, user geo-distribution, and traffic demand, among others, from a tier-1 mobile operator in the US. We evaluate CaRL on two DT scenarios representing two network clusters in two different cities and compare its performance with the business-as-usual (BAU) policy and other competing optimization approaches using heuristic and Q-table algorithms. Benchmarking results show that CaRL performs the best and improves the average cluster-aggregated downlink throughput over the BAU policy by 24% and 18% in these two scenarios, respectively.

Hierarchical Reinforcement Learning Based Traffic Steering in Multi-RAT 5G Deployments

Traffic Steering for 5G Multi-RAT Deployments using Deep Reinforcement Learning

Machine Learning-enabled Traffic Steering in O-RAN: A Case Study on Hierarchical Learning Approach

On Deep Reinforcement Learning for Traffic Steering Intelligent ORAN

Toward Scalable and Efficient Hierarchical Deep Reinforcement Learning for 5G RAN Slicing

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

Intelligent O-RAN Traffic Steering for URLLC Through Deep Reinforcement Learning

Multi-objective Resource Allocation for 5G Using Hierarchical Reinforcement Learning

Adaptive Traffic Signal Control for Large-Scale Scenario with Cooperative Group-based Multi-agent Reinforcement Learning

Intelligent Traffic Steering in Beyond 5G Open RAN based on LSTM Traffic Prediction

DRL-Based RAT Selection in a Hybrid Vehicular Communication Network

Learning from Peers: Deep Transfer Reinforcement Learning for Joint Radio and Cache Resource Allocation in 5G RAN Slicing

CaRL: Cascade Reinforcement Learning with State Space Splitting for O-RAN based Traffic Steering

Multi-RAT Access Based on Multi-Agent Reinforcement Learning

Joint Computing and Caching in 5G-Envisioned Internet of Vehicles: A Deep Reinforcement Learning-Based Traffic Control System

Deep Reinforcement Learning for Adaptive Traffic Signal Control

Deep Reinforcement Learning for Router Selection in Network with Heavy Traffic

RAN Information-Assisted TCP Congestion Control Using Deep Reinforcement Learning with Reward Redistribution

Communication-Efficient Orchestrations for URLLC Service via Hierarchical Reinforcement Learning

Multiobjective Reinforcement Learning for Traffic Signal Control Using Vehicular Ad Hoc Network

Augmented Mixed Vehicular Platoon Control with Dense Communication Reinforcement Learning for Traffic Oscillation Alleviation