Abstract:rApps and xApps need to be controlled and orchestrated well in the open radio access network (O-RAN) so that they can deliver a guaranteed network performance in a complex multi-vendor environment. This paper proposes a novel intent-driven intelligent control and orchestration scheme based on hierarchical reinforcement learning (HRL). The proposed scheme can orchestrate multiple rApps or xApps according to the operator's intent of optimizing certain key performance indicators (KPIs), such as throughput, energy efficiency, and latency. Specifically, we propose a bi-level architecture with a meta-controller and a controller. The meta-controller provides the target performance in terms of KPIs, while the controller performs xApp orchestration at the lower level. Our simulation results show that the proposed HRL-based intent-driven xApp orchestration mechanism achieves 7.5% and 21.4% increase in average system throughput with respect to two baselines, i.e., a single xApp baseline and a non-machine learning-based algorithm, respectively. Similarly, 17.3% and 37.9% increase in energy efficiency are observed in comparison to the same baselines.

What problem does this paper attempt to address?

### Problems Addressed by the Paper The paper aims to address the intelligent control and orchestration of multiple rApps and xApps in Open Radio Access Networks (O-RAN) to optimize key performance indicators (KPIs) such as throughput, energy efficiency, and latency in a multi-vendor environment. Specifically, the paper proposes an intent-driven intelligent control and orchestration scheme based on Hierarchical Reinforcement Learning (HRL). #### Main Contributions: 1. **Hierarchical Reinforcement Learning Scheme**: A hierarchical reinforcement learning method is proposed to achieve intent-driven xApps orchestration, applicable to both non-real-time RAN Intelligent Controller (non-RT-RIC) and near-real-time RAN Intelligent Controller (near-RT-RIC) in O-RAN. 2. **Intent-Driven Orchestration**: The intent of human operators is conveyed to the system as goals, enabling the orchestration of xApps to meet specified performance targets. 3. **Enhanced Flexibility**: The intent-driven approach enhances the flexibility of network control and management. #### Method Overview: - **Two-Layer Architecture**: Comprises a Meta-Controller and a Controller. The Meta-Controller is responsible for receiving the operator's intent and translating it into optimization goals; the Controller executes specific xApps orchestration tasks based on these goals. - **Three xApps Instances**: 1. **Traffic Steering xApp**: Utilizes Deep Q-Network (DQN) for traffic scheduling to balance the QoS requirements of different service types. 2. **Cell Sleeping xApp**: Reduces energy consumption by shutting down idle or low-load base stations. 3. **Beamforming xApp**: Employs intelligent beamforming techniques to improve throughput and optimize energy efficiency. #### Experimental Results: - Compared to single xApp and non-machine learning algorithms, the proposed HRL baseline scheme improves average system throughput by 7.5% and 21.4%, respectively, and energy efficiency by 17.3% and 37.9%, respectively. In summary, the paper is dedicated to enhancing the performance of O-RAN networks through intent-driven intelligent orchestration of xApps using hierarchical reinforcement learning.

Intent-driven Intelligent Control and Orchestration in O-RAN Via Hierarchical Reinforcement Learning

A Comparative Analysis of Deep Reinforcement Learning-based xApps in O-RAN

LLM-Based Intent Processing and Network Optimization Using Attention-Based Hierarchical Reinforcement Learning

PandORA: Automated Design and Comprehensive Evaluation of Deep Reinforcement Learning Agents for Open RAN

Actor-Critic Network for O-RAN Resource Allocation: xApp Design, Deployment, and Analysis

ColO-RAN: Developing Machine Learning-based xApps for Open RAN Closed-loop Control on Programmable Experimental Platforms

Federated Deep Reinforcement Learning for Resource Allocation in O-RAN Slicing

Team Learning-Based Resource Allocation for Open Radio Access Network (O-RAN)

OREO: O-RAN intElligence Orchestration of xApp-based network services

Machine Learning-enabled Traffic Steering in O-RAN: A Case Study on Hierarchical Learning Approach

Hierarchical Meta-Reinforcement Learning for Resource-Efficient Slicing in O-RAN

Machine Learning-based xApp for Dynamic Resource Allocation in O-RAN Networks

Intelligent Closed-loop RAN Control with xApps in OpenRAN Gym

Deep reinforcement learning for RAN optimization and control

User Access Control in Open Radio Access Networks: A Federated Deep Reinforcement Learning Approach

Intelligent O-RAN Traffic Steering for URLLC Through Deep Reinforcement Learning

Communication-Efficient Orchestrations for URLLC Service via Hierarchical Reinforcement Learning

Meta Reinforcement Learning Approach for Adaptive Resource Optimization in O-RAN

CaRL: Cascade Reinforcement Learning with State Space Splitting for O-RAN based Traffic Steering

Enhancing Energy Efficiency in O-RAN Through Intelligent xApps Deployment

ScalO-RAN: Energy-aware Network Intelligence Scaling in Open RAN