Abstract:Heterogeneous network (HetNet) is a promising solution to satisfy the unprecedented demand for higher data rate in the next generation mobile networks. Different from the traditional single-layer cellular networks, how to provide the best service to the user equipments (UEs) under the limited resource is an urgent problem to solve. In order to efficiently address the above challenge and strive towards high network energy efficiency, the joint optimization problem of user association and power control in orthogonal frequency division multiple access (OFDMA) based uplink HetNets is studied. Considering the non-convex and non-linear characteristics of the problem, a multi-agent deep Q-learning Network (DQN) method is studied to solve the problem. Different from the traditional methods, such as game theory, fractional programming and convex optimization, which need more and accurate network information in practice, the multi-agent DQN method requires less communication information of the environment. Moreover, for the communication environment dynamics, the maximum long-term overall network utility with a new reward function while ensuring the UE's quality of service (QoS) requirements is achieved by using the multi-agent DQN method. Then, according to the application scenario, the action space, state space and reward function of the multi-agent DQN based framework are redefined and formulated. Simulation results demonstrate that the multi-agent DQN method has the best performance on convergence and energy efficiency compared with the traditional reinforcement learning (Q-learning). (C) 2020 Elsevier B.V. All rights reserved.

A Q-Learning-Based Power Allocation Method for Two-Tier Heterogeneous Networks

Q-Learning Based Selection Strategies for Load Balance and Energy Balance in Heterogeneous Networks

User Association and Power Allocation Based on Q-Learning in Ultra Dense Heterogeneous Networks

Distributed Q-Learning Aided Heterogeneous Network Association for Energy-Efficient IIoT.

A Power Allocation Scheme Based on Deep Reinforcement Learning in HetNets.

A deep reinforcement learning for user association and power control in heterogeneous networks

Reinforcement Learning Based Dynamic Energy-Saving Algorithm For Three-Tier Heterogeneous Networks

QoS-aware energy-efficient power control in two-tier femtocell networks based on Q-learning

Energy Efficient Transmission in Underlay CR-NOMA Networks Enabled by Reinforcement Learning

Distributed Deep Reinforcement Learning-Based Spectrum and Power Allocation for Heterogeneous Networks

Joint Power Control and User Association Strategy in Green HetNets Using Deep Q-Network with LSTM

Power Allocation Based on Deep Reinforcement Learning in HetNets with Varying User Activity.

A Novel Network Selection Approach in 5G Heterogeneous Networks Using Q-Learning

Resource Allocation in HetNets with Green Energy Supply Based on Deep Reinforcement Learning

Joint Cell Activation and User Association for Backhaul Load Balancing in Green HetNets

A Fully Distributed and Clustered Learning of Power Control in User-Centric Ultra-Dense HetNets

An Energy-Aware Network Routing Algorithm Based on Q-Learning

Deep Reinforcement Learning Framework For Joint Resource Allocation In Heterogeneous Networks

Energy Efficiency Enhancement in Heterogeneous Networks: a Joint Resource Allocation Approach

Deep Q-Learning Based Dynamic Resource Allocation for Self-Powered Ultra-Dense Networks

Self-optimized Heterogeneous Networks for Energy Efficiency