Abstract:This paper proposes a multi-agent double deep Q network (DDQN)-based approach to jointly optimize the beamforming vectors and power splitting (PS) ratio in multi-user multiple-input single-output (MU-MISO) simultaneous wireless information and power transfer (SWIPT)-enabled heterogeneous networks (HetNets), where a macro base station (MBS) and several femto base stations (FBSs) serve multiple macro user equipments (MUEs) and femto user equipments (FUEs). The PS receiver architecture is deployed at FUEs. An optimization problem is formulated to maximize the achievable sum information rate of FUEs under the constraints of the achievable information rate requirements of MUEs and FUEs and the energy harvesting (EH) requirements of FUEs. Since the optimization problem is challenging to handle due to the high dimension and time-varying environment, an efficient multi-agent DDQN-based algorithm is presented, which is trained in a centralized manner and runs in a distributed manner, where two sets of deep neural network parameters are jointly updated and trained to tackle the problem and avoid overestimation. To facilitate the presented multi-agent DDQN-based algorithm, the action space, the state space and the reward function are designed, where the codebook matrix is employed to deal with the complex transmit beamforming vectors. Simulation results validate the proposed algorithm. Notable performance gains are achieved by the proposed algorithm due to considering the beam directions in the action space and the adaptability to the Doppler frequency shifts. Besides, the proposed algorithm is shown to be superior to other benchmark ones numerically.

Multibeam Antennas Handover Algorithm Based on Multi-Objective Bayesian Optimization

Handover Based on Receive Beamforming in High Mobility Cellular Communication Networks

An MDP-based Handoff Decision Algorithm for Multi-Domain Heterogeneous Wireless Access Networks

Gradual beamforming and soft handover in high mobility cellular communication networks

Robust Transceiver Optimization for Multi-Antenna Multi-Carrier Multi-Hop Communications

Multi-sector Joint Beamforming for Pilot Coverage Optimization

Multi-fidelity Bayesian algorithm for antenna optimization

An Efficient Multi-Agent Optimization Approach for Coordinated Massive MIMO Beamforming.

Multiobjective Optimization for Beam Hopping and Power Allocation in Dual Satellite Cooperative Transmission Networks

A Broad Beamforming Approach for High-Mobility Communications

Transferable Multi-Fidelity Bayesian Optimization for Radio Resource Management

Pointing Optimization of Multi-Beam Array Antenna under Interference

A novel handover scheme for millimeter wave network: An approach of integrating reinforcement learning and optimization

Movable Antenna-Aided Hybrid Beamforming for Multi-User Communications

Beam Management in Low Earth Orbit Satellite Communication With Handover Frequency Control and Satellite-Terrestrial Spectrum Sharing

Optimizing Data Transmission in High Altitude Balloon Networks with Multi-beam Directional Antennas

Cross-Layer Optimization: Joint User Scheduling and Beamforming Design with QoS Support in Joint Transmission Networks

Beam-Steering Optimization in Multi-UAVs Mmwave Networks: A Mean Field Game Approach

Tracking the Best Beam for a Mobile User via Bayesian Optimization

Optimization of Power Beacon Assisted Wireless Powered Two-Way Relaying Systems under User Fairness

Joint Coordinated Beamforming and Power Splitting Ratio Optimization in MU-MISO SWIPT-Enabled HetNets: A Multi-Agent DDQN-Based Approach