Abstract:This paper first provides a brief survey on existing traffic offloading techniques in wireless networks. Particularly as a case study, we put forward an online reinforcement learning framework for the problem of traffic offloading in a stochastic heterogeneous cellular network (HCN), where the time-varying traffic in the network can be offloaded to nearby small cells. Our aim is to minimize the total discounted energy consumption of the HCN while maintaining the quality-of-service (QoS) experienced by mobile users. For each cell (i.e., a macro cell or a small cell), the energy consumption is determined by its system load, which is coupled with system loads in other cells due to the sharing over a common frequency band. We model the energy-aware traffic offloading problem in such HCNs as a discrete-time Markov decision process (DTMDP). Based on the traffic observations and the traffic offloading operations, the network controller gradually optimizes the traffic offloading strategy with no prior knowledge of the DTMDP statistics. Such a model-free learning framework is important, particularly when the state space is huge. In order to solve the curse of dimensionality, we design a centralized Q-learning with compact state representation algorithm, which is named QC-learning. Moreover, a decentralized version of the QC-learning is developed based on the fact the macro base stations (BSs) can independently manage the operations of local small-cell BSs through making use of the global network state information obtained from the network controller. Simulations are conducted to show the effectiveness of the derived centralized and decentralized QC-learning algorithms in balancing the tradeoff between energy saving and QoS satisfaction.

User Scheduling and Resource Allocation in HetNets with Hybrid Energy Supply: an Actor-Critic Reinforcement Learning Approach

Energy Efficient Transmission in Underlay CR-NOMA Networks Enabled by Reinforcement Learning

Mobility-Aware Centralized Reinforcement Learning for Dynamic Resource Allocation in HetNets

Multi-Agent Actor Critic for Channel Allocation in Heterogeneous Networks.

Green Resource Allocation and Energy Management in Heterogeneous Small Cell Networks Powered by Hybrid Energy

Reinforcement Learning Based Content Push Policy for HetNets with Energy Harvesting Small Cells.

A Dynamic Power Allocation Scheme in Power-Domain NOMA Using Actor-Critic Reinforcement Learning.

A Power Allocation Scheme Based on Deep Reinforcement Learning in HetNets.

Energy-aware and QoS-aware Load Balancing for HetNets Powered by Renewable Energy

Multi-Objective Optimization of Energy Saving and Throughput in Heterogeneous Networks Using Deep Reinforcement Learning

Dynamic Resource Allocation for Energy-Efficiency Maximization in Heterogeneous Wireless Networks

Reinforcement Learning-Based Resource Allocation and Energy Efficiency Optimization for a Space–Air–Ground-Integrated Network

Network Resource Allocation Algorithm Using Reinforcement Learning Policy-Based Network in a Smart Grid Scenario

Deep Reinforcement Learning for Energy-Efficient Power Control in Heterogeneous Networks

Joint Optimization of BS Operation, User Association, Subcarrier Assignment, and Power Allocation for Energy-Efficient HetNets

Optimal Energy Management of Energy Internet: A Distributed Actor-Critic Reinforcement Learning Method

Deep Reinforcement Learning for Multi-Agent Power Control in Heterogeneous Networks

Deep Reinforcement Learning-Based Energy Minimization Task Offloading and Resource Allocation for Air Ground Integrated Heterogeneous Networks

Backhaul-Aware User Association and Resource Allocation for Energy-Constrained HetNets

An Actor-Critic Deep Reinforcement Learning Approach for Transmission Scheduling in Cognitive Internet of Things Systems

Energy-Efficiency Oriented Traffic Offloading in Wireless Networks: A Brief Survey and a Learning Approach for Heterogeneous Cellular Networks