Abstract:This paper first provides a brief survey on existing traffic offloading techniques in wireless networks. Particularly as a case study, we put forward an online reinforcement learning framework for the problem of traffic offloading in a stochastic heterogeneous cellular network (HCN), where the time-varying traffic in the network can be offloaded to nearby small cells. Our aim is to minimize the total discounted energy consumption of the HCN while maintaining the quality-of-service (QoS) experienced by mobile users. For each cell (i.e., a macro cell or a small cell), the energy consumption is determined by its system load, which is coupled with system loads in other cells due to the sharing over a common frequency band. We model the energy-aware traffic offloading problem in such HCNs as a discrete-time Markov decision process (DTMDP). Based on the traffic observations and the traffic offloading operations, the network controller gradually optimizes the traffic offloading strategy with no prior knowledge of the DTMDP statistics. Such a model-free learning framework is important, particularly when the state space is huge. In order to solve the curse of dimensionality, we design a centralized Q-learning with compact state representation algorithm, which is named QC-learning. Moreover, a decentralized version of the QC-learning is developed based on the fact the macro base stations (BSs) can independently manage the operations of local small-cell BSs through making use of the global network state information obtained from the network controller. Simulations are conducted to show the effectiveness of the derived centralized and decentralized QC-learning algorithms in balancing the tradeoff between energy saving and QoS satisfaction.

Small cell switch policy: A reinforcement learning approach

Energy efficient switch policy for small cells

Small cell switch policy: A consideration of start-up energy cost

Tradeoff Between Network Energy Consumption and Terminal Energy Consumption Via Small Cell Power Control

Dual-threshold Sleep Mode Control Scheme for Small Cells.

Deep reinforcement learning for base station switching scheme with federated LSTM‐based traffic predictions

DRAG: Deep Reinforcement Learning Based Base Station Activation in Heterogeneous Networks

Reinforcement Learning Based Content Push Policy for HetNets with Energy Harvesting Small Cells.

Hybrid Reinforcement Learning for Optimal Control of Non-Linear Switching System

Dynamic On/off Control of Wireless Small Cells with Heterogeneous Backhauls

To Switch or Not to Switch? Balanced Policy Switching in Offline Reinforcement Learning

Energy Saving Through a Learning Framework in Greener Cellular Radio Access Networks.

A User-Centric Load Balance Scheme for Small Cell Networks

Deep Q-Learning with Low Switching Cost

Adaptive Dynamic Programming for Energy-Efficient Base Station Cell Switching

Energy-Efficiency Oriented Traffic Offloading in Wireless Networks: A Brief Survey and a Learning Approach for Heterogeneous Cellular Networks

A Two-stage Multi-agent Deep Reinforcement Learning Method for Urban Distribution Network Reconfiguration Considering Switch Contribution

Finite Horizon Multi-Agent Reinforcement Learning in Solving Optimal Control of State-Dependent Switched Systems

Deep Reinforcement Learning Based Task Offloading and Resource Allocation in Small Cell MEC

Cell Switching in HAPS-Aided Networking: How the Obscurity of Traffic Loads Affects the Decision

Learning Buffer Management Policies for Shared Memory Switches