Abstract:The open radio access network (O-RAN) architecture supports intelligent network control algorithms as one of its core capabilities. Data-driven applications incorporate such algorithms to optimize radio access network (RAN) functions via RAN intelligent controllers (RICs). Deep reinforcement learning (DRL) algorithms are among the main approaches adopted in the O-RAN literature to solve dynamic radio resource management problems. However, despite the benefits introduced by the O-RAN RICs, the practical adoption of DRL algorithms in real network deployments falls behind. This is primarily due to the slow convergence and unstable performance exhibited by DRL agents upon deployment and when encountering previously unseen network conditions. In this paper, we address these challenges by proposing transfer learning (TL) as a core component of the training and deployment workflows for the DRL-based closed-loop control of O-RAN functionalities. To this end, we propose and design a hybrid TL-aided approach that leverages the advantages of both policy reuse and distillation TL methods to provide safe and accelerated convergence in DRL-based O-RAN slicing. We conduct a thorough experiment that accommodates multiple services, including real VR gaming traffic to reflect practical scenarios of O-RAN slicing. We also propose and implement policy reuse and distillation-aided DRL and non-TL-aided DRL as three separate baselines. The proposed hybrid approach shows at least: 7.7% and 20.7% improvements in the average initial reward value and the percentage of converged scenarios, and a 64.6% decrease in reward variance while maintaining fast convergence and enhancing the generalizability compared with the baselines.

Policy Poisoning Attacks on Transfer Learning Enabled Resource Allocation for Network Slicing

Jamming Attacks and Mitigation in Transfer Learning Enabled 5G RAN Slicing

Smart Jamming Attack and Mitigation on Deep Transfer Reinforcement Learning Enabled Resource Allocation for Network Slicing

Moving Target Defense based Secured Network Slicing System in the O-RAN Architecture

Toward Scalable and Efficient Hierarchical Deep Reinforcement Learning for 5G RAN Slicing

Learning from Peers: Deep Transfer Reinforcement Learning for Joint Radio and Cache Resource Allocation in 5G RAN Slicing

Using Deep Reinforcement Learning for 5G RAN Slicing Resource Allocation in New Power Load Management System

DDPG with Transfer Learning and Meta Learning Framework for Resource Allocation in Underlay Cognitive Radio Network

T-s3ra: traffic-aware scheduling for secure slicing and resource allocation in sdn/nfv enabled 5g networks

Self-Play Ensemble Q-learning enabled Resource Allocation for Network Slicing

Robust Network Slicing: Multi-Agent Policies, Adversarial Attacks, and Defensive Strategies

Hierarchical Reinforcement Learning Based Resource Allocation for RAN Slicing

Toward an Efficient and Dynamic Allocation of Radio Access Network Slicing Resources for 5G Era

Safe and Accelerated Deep Reinforcement Learning-based O-RAN Slicing: A Hybrid Transfer Learning Approach

RAN Resource Slicing in 5G Using Multi-Agent Correlated Q-Learning

Enhancing O-RAN Security: Evasion Attacks and Robust Defenses for Graph Reinforcement Learning-based Connection Management

Good Learning, Bad Performance: A Novel Attack Against RL-Based Congestion Control Systems

A Constrained Deep Reinforcement Learning Optimization for Reliable Network Slicing in a Blockchain-Secured Low-Latency Wireless Network

Safe and Fast Reinforcement Learning for Network Slicing Resource Allocation

Attacking Slicing Network via Side-channel Reinforcement Learning Attack

Knowledge Transfer Based Radio and Computation Resource Allocation for 5G RAN Slicing