Abstract:This paper proposes a multi-agent double deep Q network (DDQN)-based approach to jointly optimize the beamforming vectors and power splitting (PS) ratio in multi-user multiple-input single-output (MU-MISO) simultaneous wireless information and power transfer (SWIPT)-enabled heterogeneous networks (HetNets), where a macro base station (MBS) and several femto base stations (FBSs) serve multiple macro user equipments (MUEs) and femto user equipments (FUEs). The PS receiver architecture is deployed at FUEs. An optimization problem is formulated to maximize the achievable sum information rate of FUEs under the constraints of the achievable information rate requirements of MUEs and FUEs and the energy harvesting (EH) requirements of FUEs. Since the optimization problem is challenging to handle due to the high dimension and time-varying environment, an efficient multi-agent DDQN-based algorithm is presented, which is trained in a centralized manner and runs in a distributed manner, where two sets of deep neural network parameters are jointly updated and trained to tackle the problem and avoid overestimation. To facilitate the presented multi-agent DDQN-based algorithm, the action space, the state space and the reward function are designed, where the codebook matrix is employed to deal with the complex transmit beamforming vectors. Simulation results validate the proposed algorithm. Notable performance gains are achieved by the proposed algorithm due to considering the beam directions in the action space and the adaptability to the Doppler frequency shifts. Besides, the proposed algorithm is shown to be superior to other benchmark ones numerically.

Power entangling and matching in cognitive wireless mesh networks by applying conjecture based multi-agent QQ-learning approach

Power Allocation for Cognitive Wireless Mesh Networks by Applying Multi-agent Q-learning Approach

Applying multi-agent Q-learning scheme in cognitive wireless mesh networks for green communications

Stochastic Power Adaptation with Multiagent Reinforcement Learning for Cognitive Wireless Mesh Networks

Green transmit power assignment for cognitive radio networks by applying multi-agent Q-learning approach

Reinforcement Learning Enhanced Iterative Power Allocation in Stochastic Cognitive Wireless Mesh Networks

Conjectural Variations in Multi-Agent Reinforcement Learning for Energy-Efficient Cognitive Wireless Mesh Networks.

Distributed iterative power allocation in cognitive wireless mesh networks

Multi-Agent Reinforcement Learning Based Unlicensed Resource Sharing for LTE-U Networks.

Energy-efficient power allocation in cognitive sensor networks: A game theoretic approach

Analysis and Optimization of Power Control in Multiuser Cognitive Wireless Networks

Multi-Agent Reinforcement Learning for Dynamic Spectrum Access.

Joint Subchannel, Rate and Power Allocation in OFDMA-Based Cognitive Wireless Mesh Network

QoS Optimization for Mobile Ad Hoc Cloud: A Multi-Agent Independent Learning Approach

A Multi-Agent Multi-Environment Mixed Q-Learning for Partially Decentralized Wireless Network Optimization

Joint Coordinated Beamforming and Power Splitting Ratio Optimization in MU-MISO SWIPT-Enabled HetNets: A Multi-Agent DDQN-Based Approach

Multi-agent Q-Learning of Channel Selection in Multi-user Cognitive Radio Systems: A Two by Two Case

Energy-Efficient Cooperative Transmission in Ultra-Dense Millimeter-Wave Network: Multi-Agent Q-Learning Approach

Heterogeneous Wireless Resources On-demand Access Distributed Decision Based on Spectrum Sensing

QoS-aware energy-efficient power control in two-tier femtocell networks based on Q-learning

A Self-Organizing Resource Allocation Strategy Based on Q-learning Approach in Ultra-Dense Networks