Abstract:In this article, we focus on a downlink cellular network, where multiple unmanned aerial vehicles (UAVs) serve as aerial base stations for ground users through frequency-division multiple access (FDMA). With user locations and channel parameters inaccessible, the UAVs coordinate to make a decision on resource allocation and trajectory design in a decentralized way. Aiming at optimizing both overall and fairness throughput, we model resource allocation and trajectory design as a decentralized partially observable Markov decision process (Dec-POMDP) and propose multiagent reinforcement learning (RL) as a solution. Specifically, we use parameterized deep $Q$ -network (P-DQN) for the action space comprising both discrete and continuous actions and the QMIX framework is leveraged to aggregate each UAV’s local critics. For fairness throughput optimization, we introduce an entropy-like fairness indicator to the reward to make the total return decomposable. In addition, we further propose a novel distributed learning framework for overall throughput optimization such that each UAV can contribute its local gradient, and model training can be implemented in parallel without need of observation data sharing among the UAVs. Simulation results show that the proposed multiagent RL approach as well as the distributed learning framework are efficient in model training and present acceptable performance close to that achieved by deterministic optimization, which relies on convention optimization techniques with user locations and channel parameters explicitly known beforehand. For fairness throughput optimization, we also show that ground users achieve individual throughputs close to each other, which verifies the effectiveness of the proposed fairness indicator as the reward definition in the RL framework.

Deep Reinforcement Learning Based Resource Allocation and Trajectory Planning in Integrated Sensing and Communications UAV Network

Multi-UAV Trajectory Design and Power Control Based on Deep Reinforcement Learning.

Dynamic Role Switching Scheme with Joint Trajectory and Power Control for Multi-UAV Cooperative Secure Communication

Deep Reinforcement Learning for Joint Trajectory Planning, Transmission Scheduling, and Access Control in UAV-Assisted Wireless Sensor Networks

Trajectory Design and Resource Allocation for Multi-UAV Networks: Deep Reinforcement Learning Approaches

Deep Reinforcement Learning Based Trajectory Design and Resource Allocation for UAV-Assisted Communications

Multi-Agent Low-Bias Reinforcement Learning for Resource Allocation in UAV-Assisted Networks

Blocklength Allocation and Power Control in UAV-Assisted URLLC System via Multi-agent Deep Reinforcement Learning

Resource Allocation and Trajectory Design in UAV-Aided Cellular Networks Based on Multiagent Reinforcement Learning

Resource Allocation in UAV-D2D Networks: A Scalable Heterogeneous Multi-Agent Deep Reinforcement Learning Approach

Deep Reinforcement Learning Based Resource Allocation in Multi-UAV-Aided MEC Networks.

Reinforcement Learning-Based UAVs Resource Allocation for Integrated Sensing and Communication (ISAC) System

Energy-Efficient Multi-UAVs Cooperative Trajectory Optimization for Communication Coverage: An MADRL Approach

Decentralized Trajectory and Power Control Based on Multi-Agent Deep Reinforcement Learning in UAV Networks

Resource Allocation in UAV-Assisted Networks: A Clustering-Aided Reinforcement Learning Approach

A deep reinforcement learning framework and its implementation for UAV-aided covert communication

Joint Optimization of Multi-UAV Deployment and User Association Via Deep Reinforcement Learning for Long-Term Communication Coverage

Resource Allocation and Trajectory Optimization in Multi-UAV Collaborative Vehicular Networks: an Extended Multi-Agent DRL Approach

Deep Reinforcement Learning Based 3D UAV Trajectory Design and Frequency Band Allocation.

Trajectory Design and Bandwidth Assignment for UAVs-enabled Communication Network with Multi - Agent Deep Reinforcement Learning.

Delay-Tolerant Multi-Agent DRL for Trajectory Planning and Transmission Control in UAV-Assisted Wireless Networks