Abstract:Artificial intelligence (AI) and Machine Learning (ML) are considered as key enablers for realizing the full potential of fifth-generation (5G) and beyond mobile networks, particularly in the context of resource management and orchestration. In this demonstration, we consider a fully-fledged 5G mobile network and develop a multi-agent deep reinforcement learning (DRL) framework for RAN resource allocation. By leveraging local monitoring information generated by a shared gNodeB instance (gNB), each DRL agent aims to optimally allocate radio resources concerning service-specific traffic demands belonging to heterogeneous running services. We perform experiments on the deployed testbed in real-time, showing that DRL-based agents can allocate radio resources fairly while improving the overall efficiency of resource utilization and minimizing the risk of over provisioning.

What problem does this paper attempt to address?

This paper attempts to address the problem of efficiently managing and allocating Radio Access Network (RAN) resources in the Fifth Generation (5G) and future mobile networks. Specifically, the authors focus on how to meet the performance requirements of emerging use cases in highly heterogeneous network environments, particularly the stringent quality of service and low latency demands. To this end, they propose a Multi-Agent Deep Reinforcement Learning (MADRL) based approach for dynamic RAN resource allocation. ### Main Issues: 1. **Resource Management and Optimization**: How to efficiently allocate and utilize wireless resources while meeting different service demands and avoiding over-provisioning of resources. 2. **Network Slicing Management**: How to achieve dynamic resource allocation for network slicing in support of large-scale network slicing scenarios, with the support of open architectures and interfaces. 3. **Real-time Decision Making**: How to make real-time resource allocation decisions in edge networks, reducing reliance on centralized controllers and lowering the overhead of control information exchange. ### Solution: - **Multi-Agent Deep Reinforcement Learning Framework**: Each agent makes independent resource allocation decisions based on local monitoring information and collaborates through a federated learning mechanism, sharing information to improve the efficiency and accuracy of overall decision-making. - **Distributed Learning Mechanism**: Agents directly allocate resources in edge networks using the latest local information, reducing dependence on the core network. - **Federated Learning Layer**: Aggregates multiple local decision models through federated learning to obtain global knowledge, further enhancing the generalization capability of decisions. ### Experimental Validation: - **Testbed Architecture**: Includes components such as AMARI UE Simbox, AMARI Callbox, monitoring systems, and local decision agents to simulate and test the performance of the MADRL framework. - **Experimental Results**: Real-time experiments validated the fairness and efficiency of the MADRL approach in resource allocation, demonstrating its application effectiveness in different network slices. In summary, this paper aims to address the challenges of resource management in 5G and future mobile networks through the MADRL approach, improving resource utilization and network performance.

A Multi-Agent Deep Reinforcement Learning Approach for RAN Resource Allocation in O-RAN

Multi-objective Resource Allocation for 5G Using Hierarchical Reinforcement Learning

Toward Scalable and Efficient Hierarchical Deep Reinforcement Learning for 5G RAN Slicing

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

DRL-based Joint Resource Scheduling of eMBB and URLLC in O-RAN

Hierarchical Reinforcement Learning Based Resource Allocation for RAN Slicing

Attention-based Open RAN Slice Management using Deep Reinforcement Learning

Meta Reinforcement Learning Approach for Adaptive Resource Optimization in O-RAN

PandORA: Automated Design and Comprehensive Evaluation of Deep Reinforcement Learning Agents for Open RAN

Dynamics of Resource Allocation in O-RANs: An In-depth Exploration of On-Policy and Off-Policy Deep Reinforcement Learning for Real-Time Applications

Real-Time Resource Slicing for 5G RAN Via Deep Reinforcement Learning

Design and Evaluation of Deep Reinforcement Learning for Energy Saving in Open RAN

Management and Orchestration of Virtual Network Functions via Deep Reinforcement Learning

DRL-based Latency-Aware Network Slicing in O-RAN with Time-Varying SLAs

On the Specialization of FDRL Agents for Scalable and Distributed 6G RAN Slicing Orchestration

Multi-Agent Deep Reinforcement Learning Joint Beamforming for Slicing Resource Allocation

Deep reinforcement learning for RAN optimization and control

Multi-Agent Deep Reinforcement Learning Based Resource Allocation for Ultra-Reliable Low-Latency Internet of Controllable Things

Demo: Deep Reinforcement Learning for Resource Management in Cellular Network Slicing.

Deep Reinforcement Learning (DRL)-based Resource Management in Software-Defined and Virtualized Vehicular Ad Hoc Networks

Deep Reinforcement Learning Based Mode Selection and Resource Management for Green Fog Radio Access Networks