Abstract:Massive random access of devices in the emerging Open Radio Access Network (O-RAN) brings great challenge to the access control and management. Exploiting the bursting nature of the access requests, sparse active user detection (SAUD) is an efficient enabler towards efficient access management, but the sparsity might be deteriorated in case of uncoordinated massive access requests. To dynamically preserve the sparsity of access requests, a reinforcement-learning (RL)-assisted scheme of closed-loop access control utilizing the access class barring technique is proposed, where the RL policy is determined through continuous interaction between the RL agent, i.e., a next generation node base (gNB), and the environment. The proposed scheme can be implemented by the near-real-time RAN intelligent controller (near-RT RIC) in O-RAN, supporting rapid switching between heterogeneous vertical applications, such as mMTC and uRLLC services. Moreover, a data-driven scheme of deep-RL-assisted SAUD is proposed to resolve highly complex environments with continuous and high-dimensional state and action spaces, where a replay buffer is applied for automatic large-scale data collection. An actor-critic framework is formulated to incorporate the strategy-learning modules into the near-RT RIC. Simulation results show that the proposed schemes can achieve superior performance in both access efficiency and user detection accuracy over the benchmark scheme for different heterogeneous services with massive access requests.

What problem does this paper attempt to address?

### Problems Addressed by the Paper The paper aims to address the challenges of access control and management brought by a large number of random device accesses in Open Radio Access Network (O-RAN). Specifically, the paper focuses on the following issues: 1. **Management of Large-Scale Random Access Requests**: - In O-RAN, a large number of devices randomly access the network, which poses significant challenges to access control and management. This challenge is particularly pronounced when multiple heterogeneous services coexist. 2. **Performance Degradation of Sparse Active User Detection (SAUD)**: - Although the connection requests of large-scale users are bursty and sparse, the sparsity may be disrupted when the access request volume surges or multiple heterogeneous services are supported simultaneously, leading to a decline in SAUD performance. 3. **Dynamically Maintaining the Sparsity of Access Requests**: - It is necessary to design an effective mechanism to dynamically manage user access requests to maintain the sparsity of access requests, thereby improving the accuracy and efficiency of SAUD. ### Solutions To address the above issues, the paper proposes the following solutions: 1. **Reinforcement Learning-Based Access Control Scheme (RL-SAUD)**: - Utilizing Reinforcement Learning (RL) technology, the scheme dynamically adjusts the Access Class Barring (ACB) factor through continuous interaction with the environment to maintain the sparsity of access requests. The RL-SAUD algorithm learns the optimal decision strategy through the Q-learning method. 2. **Deep Reinforcement Learning-Based SAUD Scheme (DRL-SAUD)**: - For complex environments with high-dimensional continuous state and action spaces, a DRL-based SAUD scheme is proposed. This scheme adopts the Actor-Critic framework, using a replay buffer to automatically collect large-scale data and accelerate the convergence speed. ### Main Contributions - **Large-Scale Access Control Framework**: - Based on ACB and SAUD, a large-scale access control framework deployed on the near-real-time RAN Intelligent Controller (near-RT RIC) in O-RAN is proposed, fully utilizing the sparsity of access requests to achieve efficient and accurate active user detection. - **RL-Assisted SAUD**: - Dynamic access management is achieved through the Q-learning method, adaptively adjusting the ACB factor to maintain the sparsity of access requests, thereby maintaining the accuracy of SAUD. - **DRL-Assisted SAUD**: - By incorporating the Actor-Critic module, the scheme uses deep neural networks to address quantization errors and performance limitations in high-dimensional state-action spaces, achieving faster convergence speeds and better access control strategies. ### Conclusion The paper verifies the superior performance of the proposed schemes in terms of access efficiency and user detection accuracy under different heterogeneous services through theoretical analysis and simulation experiments. These schemes provide effective methods for large-scale access management and dynamic control in O-RAN.

Sparsity-Aware Intelligent Massive Random Access Control in Open RAN: A Reinforcement Learning Based Approach

User Access Control in Open Radio Access Networks: A Federated Deep Reinforcement Learning Approach

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

Federated Deep Reinforcement Learning for User Access Control in Open Radio Access Networks

Deep Reinforcement Learning for Multi-User Access Control in Non-Terrestrial Networks.

Attention-based Open RAN Slice Management using Deep Reinforcement Learning

Access Control for RAN Slicing Based on Federated Deep Reinforcement Learning

O-RAN architecture, interfaces, and standardization: Study and application to user intelligent admission control

Intelligent Access to Unlicensed Spectrum: A Mean Field Based Deep Reinforcement Learning Approach

A Decentralized Pilot Assignment Algorithm for Scalable O-RAN Cell-Free Massive MIMO

Delay-Aware Massive Random Access for Machine-Type Communications Via Hierarchical Stochastic Learning

DDPG with Transfer Learning and Meta Learning Framework for Resource Allocation in Underlay Cognitive Radio Network

Team Learning-Based Resource Allocation for Open Radio Access Network (O-RAN)

Reinforcement Learning for Improved Random Access in Delay-Constrained Heterogeneous Wireless Networks

Intelligent O-RAN Traffic Steering for URLLC Through Deep Reinforcement Learning

Reinforcement Learning Random Access for Delay-Constrained Heterogeneous Wireless Networks: A Two-User Case

A Multi-Agent Deep Reinforcement Learning Approach for RAN Resource Allocation in O-RAN

DRL-based Joint Resource Scheduling of eMBB and URLLC in O-RAN

Dynamic multiple access based on deep reinforcement learning for Internet of Things

A Reinforcement Learning Framework for Autonomous Cell Activation and Customized Energy-Efficient Resource Allocation in C-RANs.

Deep Reinforcement Learning-Based Multi-Access in Massive Machine-Type Communication