Abstract:This paper proposes a multi-agent double deep Q network (DDQN)-based approach to jointly optimize the beamforming vectors and power splitting (PS) ratio in multi-user multiple-input single-output (MU-MISO) simultaneous wireless information and power transfer (SWIPT)-enabled heterogeneous networks (HetNets), where a macro base station (MBS) and several femto base stations (FBSs) serve multiple macro user equipments (MUEs) and femto user equipments (FUEs). The PS receiver architecture is deployed at FUEs. An optimization problem is formulated to maximize the achievable sum information rate of FUEs under the constraints of the achievable information rate requirements of MUEs and FUEs and the energy harvesting (EH) requirements of FUEs. Since the optimization problem is challenging to handle due to the high dimension and time-varying environment, an efficient multi-agent DDQN-based algorithm is presented, which is trained in a centralized manner and runs in a distributed manner, where two sets of deep neural network parameters are jointly updated and trained to tackle the problem and avoid overestimation. To facilitate the presented multi-agent DDQN-based algorithm, the action space, the state space and the reward function are designed, where the codebook matrix is employed to deal with the complex transmit beamforming vectors. Simulation results validate the proposed algorithm. Notable performance gains are achieved by the proposed algorithm due to considering the beam directions in the action space and the adaptability to the Doppler frequency shifts. Besides, the proposed algorithm is shown to be superior to other benchmark ones numerically.

A deep reinforcement learning for user association and power control in heterogeneous networks

Deep Reinforcement Learning for User Association and Resource Allocation in Heterogeneous Cellular Networks

A Power Allocation Scheme Based on Deep Reinforcement Learning in HetNets.

Parallel Deep Reinforcement Learning Based Online User Association Optimization in Heterogeneous Networks.

Deep Reinforcement Learning for Multi-Agent Power Control in Heterogeneous Networks

A Distributed User Association Method for LTE-U by Enabling Q-Learning in Minority Game.

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

Multi-Agent Reinforcement Learning Based Unlicensed Resource Sharing for LTE-U Networks.

Joint Optimization of Handover Control and Power Allocation Based on Multi-Agent Deep Reinforcement Learning

Deep Reinforcement Learning for Energy-Efficient Power Control in Heterogeneous Networks

Multicell Power Control under QoS Requirements with CNet

Joint Deep Reinforcement Learning and Unsupervised Learning for Channel Selection and Power Control in D2D Networks

Distributed Deep Reinforcement Learning-Based Spectrum and Power Allocation for Heterogeneous Networks

Distributed Q-Learning Aided Heterogeneous Network Association for Energy-Efficient IIoT.

Joint Interference Alignment and Power Control for Dense Networks Via Deep Reinforcement Learning

Power Allocation in Multi-User Cellular Networks: Deep Reinforcement Learning Approaches

DQN-Based Multi-User Power Allocation for Hybrid RF/VLC Networks

Deep Reinforcement Learning Based Resource Allocation for Heterogeneous Networks

A Self-Organizing Resource Allocation Strategy Based on Q-learning Approach in Ultra-Dense Networks

Dynamic User Pairing and Power Allocation for NOMA with Deep Reinforcement Learning

Joint Coordinated Beamforming and Power Splitting Ratio Optimization in MU-MISO SWIPT-Enabled HetNets: A Multi-Agent DDQN-Based Approach