Abstract:The intensive deployment of sixth-generation (6G) base stations is expected to greatly enhance network service capabilities, offering significantly higher throughput and lower latency compared to previous generations. However, this advancement is accompanied by a notable increase in the number of network elements, leading to increased power consumption. This not only worsens carbon emissions but also significantly raises operational costs for network operators. To address the challenges arising from this surge in network energy consumption, there is a growing focus on innovative energy-saving technologies designed for 6G networks. These technologies involve strategies for dynamically adjusting the operational status of base stations, such as activating sleep modes during periods of low demand, to optimize energy use while maintaining network performance and efficiency. Furthermore, integrating artificial intelligence into the network's operational framework is being explored to establish a more energy-efficient, sustainable, and cost-effective 6G network. In this paper, we propose a small base station sleeping control scheme in heterogeneous dense small cell networks based on federated reinforcement learning, which enables the small base stations to dynamically enter appropriate sleep modes, to reduce power consumption while ensuring users' quality-of-service (QoS) requirements. In our scheme, double deep Q-learning is used to solve the complex non-convex base station sleeping control problem. To tackle the dynamic changes in QoS requirements caused by user mobility, small base stations share local models with the macro base station, which acts as the central control unit, via the X2 interface. The macro base station aggregates local models into a global model and then distributes the global model to each base station for the next round of training. By alternately performing model training, aggregation, and updating, each base station in the network can dynamically adapt to changes in QoS requirements brought about by user mobility. Simulations show that compared with methods based on distributed deep Q-learning, our proposed scheme effectively reduces the performance fluctuations caused by user handover and achieves lower network energy consumption while guaranteeing users' QoS requirements.

Transfer Learning for QoS Aware Topology Management in Energy Efficient 5G Cognitive Radio Networks

Two-Timescale Qos-Aware Cross-Layer Optimisation For Hetnets With Flexible Backhaul

DDPG with Transfer Learning and Meta Learning Framework for Resource Allocation in Underlay Cognitive Radio Network

Transfer Reinforcement Learning for 5G-NR mm-Wave Networks

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

Learning from Peers: Deep Transfer Reinforcement Learning for Joint Radio and Cache Resource Allocation in 5G RAN Slicing

Toward Scalable and Efficient Hierarchical Deep Reinforcement Learning for 5G RAN Slicing

Reinforcement Learning-Based Computing and Transmission Scheduling for LTE-U-Enabled IoT

Q-Learning-Based Spectrum Access for Multimedia Transmission Over Cognitive Radio Networks

TACT: A Transfer Actor-Critic Learning Framework for Energy Saving in Cellular Radio Access Networks.

Using Reinforcement Learning to Reduce Energy Consumption of Ultra-Dense Networks With 5G Use Cases Requirements

Deep Reinforcement Learning for Network Energy Saving in 6G and Beyond Networks

Q-learning Based Resource Allocation for Hybrid Services with Self-similar Traffic

Human Behavior Based Learning Framework for Spectrum Allocation in Downlink CoMP for 5G Networks

Dynamic Multi-Sleeping Control with Diverse Quality-of-Service Requirements in Sixth-Generation Networks Using Federated Learning

Cognitive network management with optimization using network protocol and machine learning model

Energy-Efficiency Oriented Traffic Offloading in Wireless Networks: A Brief Survey and a Learning Approach for Heterogeneous Cellular Networks

Deep Q-learning multiple networks based dynamic spectrum access with energy harvesting for green cognitive radio network

Energy-efficient traffic offloading for RSMA-based hybrid satellite terrestrial networks with deep reinforcement learning

On-Device Intelligence for 5G RAN: Knowledge Transfer and Federated Learning enabled UE-Centric Traffic Steering

Learning-Based Energy-Efficient Resource Management by Heterogeneous RF/VLC for Ultra-Reliable Low-Latency Industrial IoT Networks