Abstract:In this article, we focus on a downlink cellular network, where multiple unmanned aerial vehicles (UAVs) serve as aerial base stations for ground users through frequency-division multiple access (FDMA). With user locations and channel parameters inaccessible, the UAVs coordinate to make a decision on resource allocation and trajectory design in a decentralized way. Aiming at optimizing both overall and fairness throughput, we model resource allocation and trajectory design as a decentralized partially observable Markov decision process (Dec-POMDP) and propose multiagent reinforcement learning (RL) as a solution. Specifically, we use parameterized deep $Q$ -network (P-DQN) for the action space comprising both discrete and continuous actions and the QMIX framework is leveraged to aggregate each UAV’s local critics. For fairness throughput optimization, we introduce an entropy-like fairness indicator to the reward to make the total return decomposable. In addition, we further propose a novel distributed learning framework for overall throughput optimization such that each UAV can contribute its local gradient, and model training can be implemented in parallel without need of observation data sharing among the UAVs. Simulation results show that the proposed multiagent RL approach as well as the distributed learning framework are efficient in model training and present acceptable performance close to that achieved by deterministic optimization, which relies on convention optimization techniques with user locations and channel parameters explicitly known beforehand. For fairness throughput optimization, we also show that ground users achieve individual throughputs close to each other, which verifies the effectiveness of the proposed fairness indicator as the reward definition in the RL framework.

Joint Channel and Power Assignment for UAV Swarm Communication Based on Multi-agent DRL

Multi-Agent Reinforcement Learning Based UAV Swarm Communications Against Jamming

Matching combined multi-agent reinforcement learning for uav secure data dissemination

Blocklength Allocation and Power Control in UAV-Assisted URLLC System via Multi-agent Deep Reinforcement Learning

Joint UAV trajectory and communication design with heterogeneous multi-agent reinforcement learning

Dynamic Role Switching Scheme with Joint Trajectory and Power Control for Multi-UAV Cooperative Secure Communication

Collaborative Decision-making in Heterogeneous UAV Swarms Based on Multi-agent Deep Reinforcement Learning

Cooperative Secure Communication in UAV Assisted Networks with Dynamic Role Switching.

Dynamic Channel Allocation for Multi-UAVs: A Deep Reinforcement Learning Approach

Multi-Agent Low-Bias Reinforcement Learning for Resource Allocation in UAV-Assisted Networks

Mean-Field Multi-Agent Reinforcement Learning for UAV Assisted Secure Data Dissemination.

Joint Resource Optimization for UAV Swarm Communication System Based on DQN

Multiobjective Deep Reinforcement Learning Based Joint Beamforming and Power Allocation in UAV Assisted Cellular Communication

Multi-Agent DRL for Air-to-Ground Communication Planning in UAV-Enabled IoT Networks

Three-Dimensional Trajectory and Resource Allocation Optimization in Multi-Unmanned Aerial Vehicle Multicast System: A Multi-Agent Reinforcement Learning Method

Resource Allocation and Trajectory Design in UAV-Aided Cellular Networks Based on Multiagent Reinforcement Learning

Joint Channel Assignment and Power Allocation for Multi-UAVs Communication Systems

Distributed UAV-BSs Trajectory Optimization for User-Level Fair Communication Service with Multi-Agent Deep Reinforcement Learning

A Novel Joint DRL-Based Utility Optimization for UAV Data Services

Deep Reinforcement Learning for Channel and Power Allocation in UAV-enabled IoT Systems

Multi-objective Deep Reinforcement Learning Based Joint Beamforming and Power Allocation in UAV Assisted Cellular Communication