Abstract:Next-generation wireless systems, already widely deployed, are expected to become even more prevalent in the future, representing challenges in both environmental and economic terms. This paper focuses on improving the energy efficiency of intelligent and programmable Open Radio Access Network (RAN) systems through the near-real-time dynamic activation and deactivation of Base Station (BS) Radio Frequency (RF) frontends using Deep Reinforcement Learning (DRL) algorithms, i.e., Proximal Policy Optimization (PPO) and Deep Q-Network (DQN). These algorithms run on the RAN Intelligent Controllers (RICs), part of the Open RAN architecture, and are designed to make optimal network-level decisions based on historical data without compromising stability and performance. We leverage a rich set of Key Performance Measurements (KPMs), serving as state for the DRL, to create a comprehensive representation of the RAN, alongside a set of actions that correspond to some control exercised on the RF frontend. We extend ns-O-RAN, an open-source, realistic simulator for 5G and Open RAN built on ns-3, to conduct an extensive data collection campaign. This enables us to train the agents offline with over 300,000 data points and subsequently evaluate the performance of the trained models. Results show that DRL agents improve energy efficiency by adapting to network conditions while minimally impacting the user experience. Additionally, we explore the trade-off between throughput and energy consumption offered by different DRL agent designs.

Energy Saving Through a Learning Framework in Greener Cellular Radio Access Networks.

TACT: A Transfer Actor-Critic Learning Framework for Energy Saving in Cellular Radio Access Networks.

Collaborative Multi-BS Power Management for Dense Radio Access Network using Deep Reinforcement Learning

Intelligent base station management in greener traffic-aware cellular networks

A Dynamic Power Allocation Scheme in Power-Domain NOMA Using Actor-Critic Reinforcement Learning.

Energy-efficient cellular network operation via base station cooperation

Energy Optimization in Ultra-Dense Radio Access Networks via Traffic-Aware Cell Switching

Energy Saving in Cellular Networks by Dynamic RS–BS Association and BS Switching

Design and Evaluation of Deep Reinforcement Learning for Energy Saving in Open RAN

Adaptive Dynamic Programming for Energy-Efficient Base Station Cell Switching

Optimizing Cellular Networks Enabled with Renewal Energy via Strategic Learning

A Reinforcement Learning Framework for Autonomous Cell Activation and Customized Energy-Efficient Resource Allocation in C-RANs.

Machine Learning Based Energy Saving Scheme in Wireless Access Networks.

Optimal Base Station Sleeping in Green Cellular Networks: A Distributed Cooperative Framework Based on Game Theory

Flexible Base Station Sleeping and Resource Cooperation Enabled Green Fully-Decoupled RAN

Adaptive transmission scheduling over fading channels for energy-efficient cognitive radio networks by reinforcement learning

Deep Reinforcement Learning Method for Energy Efficient Resource Allocation in Next Generation Wireless Networks.

Improving energy efficiency in Green femtocell networks: A hierarchical reinforcement learning framework

On Improving Energy Efficiency within Green Femtocell Networks: A Hierarchical Reinforcement Learning Approach

Minimizing Energy Cost of Base Stations with Consideration of Switching on/off Cost

An Adaptive Cell Switch Off framework to Increase Energy Efficiency in Cellular Networks