Abstract:This paper proposes a novel design on the wireless powered communication network (WPCN) in dynamic environments under the assistance of multiple unmanned aerial vehicles (UAVs). Unlike the existing studies, where the low-power wireless nodes (WNs) often conform to the coherent harvest-then-transmit protocol, under our newly proposed double-threshold based WN type updating rule, each WN can dynamically and repeatedly update its WN type as an E-node for non-linear energy harvesting over time slots or an I-node for transmitting data over sub-slots. To maximize the total transmission data size of all the WNs over T slots, each of the UAVs individually determines its trajectory and binary wireless energy transmission (WET) decisions over times slots and its binary wireless data collection (WDC) decisions over sub-slots, under the constraints of each UAV's limited on-board energy and each WN's node type updating rule. However, due to the UAVs' tightly-coupled trajectories with their WET and WDC decisions, as well as each WN's time-varying battery energy, this problem is difficult to solve optimally. We then propose a new multi-agent based hierarchical deep reinforcement learning (MAHDRL) framework with two tiers to solve the problem efficiently, where the soft actor critic (SAC) policy is designed in tier-1 to determine each UAV's continuous trajectory and binary WET decision over time slots, and the deep-Q learning (DQN) policy is designed in tier-2 to determine each UAV's binary WDC decisions over sub-slots under the given UAV trajectory from tier-1. Both of the SAC policy and the DQN policy are executed distributively at each UAV. Finally, extensive simulation results are provided to validate the outweighed performance of the proposed MAHDRL approach over various state-of-the-art benchmarks.

Power Allocation Based on Multi-Agent Deep Deterministic Policy Gradient for Underwater Acoustic Communication Networks

Multi-Agent Reinforcement Learning Based UAV Swarm Communications Against Jamming

Power Allocation for Full-Duplex Communication Systems Based on Deep Deterministic Policy Gradient

Game-based Hierarchical Multi-Armed Bandit Learning Algorithm for Joint Channel and Power Allocation in Underwater Acoustic Communication Networks

Multi-agent deep reinforcement learning based multiple access for underwater cognitive acoustic sensor networks

Exploiting Propagation Delay in Underwater Acoustic Communication Networks via Deep Reinforcement Learning

Air-Ground Coordination Communication by Multi-Agent Deep Reinforcement Learning

Penalized Reinforcement Learning-Based Energy-Efficient UAV-RIS Assisted Maritime Uplink Communications Against Jamming

Joint resource allocation in underwater acoustic communication networks: A game-based hierarchical adversarial multiplayer multiarmed bandit algorithm.

Learning Effective Communication for Cooperative Pursuit with Multi-Agent Reinforcement Learning

UAV-Enabled Secure Communications by Multi-Agent Deep Reinforcement Learning

Multi-Objective Optimization in Air-to-Air Communication System Based on Multi-Agent Deep Reinforcement Learning

A Hierarchical Deep-Reinforcement Approach

User Association and Power Allocation for User-Centric Smart-Duplex Networks via Tree-Structured Deep Reinforcement Learning

Adaptive Switching for Multimodal Underwater Acoustic Communications Based on Reinforcement Learning

HA-MARL: Heuristic and APF Assisted Multi-Agent Reinforcement Learning for Wireless Data Sharing in AUV Swarms

MAB-based Two-Tier Learning Algorithms for Joint Channel and Power Allocation in Stochastic Underwater Acoustic Communication Networks

Resource Allocation in UAV-D2D Networks: A Scalable Heterogeneous Multi-Agent Deep Reinforcement Learning Approach

Blocklength Allocation and Power Control in UAV-Assisted URLLC System via Multi-agent Deep Reinforcement Learning

On Designing Multi-UAV aided Wireless Powered Dynamic Communication via Hierarchical Deep Reinforcement Learning

Deep Reinforcement Learning for Multi-Agent Power Control in Heterogeneous Networks