Abstract:As an aerial base station, unmanned aerial vehicle (UAV) has been considered as a promising technology to assist future wireless communications due to its flexible, swift and low cost features, where resource allocation is the basis for ensuring energy-efficient UAV-assisted networks. This paper formulates a joint optimization problem of user association, UAV trajectory design and power control to maximize the channel capacity among all ground users at a limited power level in a downlink transmission. To tackle the mixed-integer non-linear programming problem, this paper proposes a clustering-aided reinforcement learning approach consisting of three consecutive stages. Firstly, modified expectation-maximization unsupervised learning algorithm is investigated to cluster the ground users, which reduces the dimensions and hence, the association complexity is reduced as well. Then, Kuhn-Munkres algorithm is incorporated for user association, which associates a UAV with the ground users via matching to the cluster, and assigns the UAVs to the centroid of the matching cluster for pre-placement, with the aim of speeding up the convergence of the following deep reinforcement learning algorithm. Finally, a multi-agent twin delayed deep deterministic (MATD3) policy gradient is proposed to solve the non-convex sub-problem, which determines the transmit power and designs the fine-tuned trajectory of UAVs. By incorporating low-bias value estimation technique, the reward of the proposed MATD3 algorithm is improved. Simulation results have demonstrated that our proposed approach achieves higher reward as well as converging faster than existing reinforcement algorithms. Besides, the clustering-aided reinforcement learning has lower computational complexity than the benchmark schemes.

Intelligently Joint Task Assignment and Trajectory Planning for UAV Cluster With Limited Communication

Onboard Distributed Trajectory Planning Through Intelligent Search for Multi-UAV Cooperative Flight

Multi-UAV Adaptive Cooperative Formation Trajectory Planning Based on an Improved MATD3 Algorithm of Deep Reinforcement Learning

Conflict resolution in multi-UAV cooperative tasks assignment with communication delay

Task Allocation and Trajectory Planning for Collaborative Adversarial Multi-UAVsystems

Cooperative Multi-UAV Path Planning with Communication Constraints

Path Planning of UAVs Based on Improved Clustering Algorithm and Ant Colony System Algorithm

Secure Multi-UAV Collaborative Task Allocation.

Distributed Cooperative Control for Deployment and Task Allocation of Unmanned Aerial Vehicle Networks

Joint User Scheduling and UAV Trajectory Optimization for Full-Duplex UAV Relaying

Path Planning of UAVs Based on Improved Ant Colony System

AoI Optimal Trajectory Planning for Cooperative UAVs: A Multi-Agent Deep Reinforcement Learning Approach

Cooperative Task Assignment of Multi-Uav System

Multi-UAV Collaborative Trajectory Planning for Seamless Data Collection and Transmission

Collaborative task assignment scheme for multi-UAV based on cluster structure

Deep Reinforcement Learning for Joint Trajectory Planning, Transmission Scheduling, and Access Control in UAV-Assisted Wireless Sensor Networks

Joint UAV trajectory and communication design with heterogeneous multi-agent reinforcement learning

On collaborative multi-UAV trajectory planning for data collection

Resource Allocation in UAV-Assisted Networks: A Clustering-Aided Reinforcement Learning Approach

Energy-Efficient Multi-UAVs Cooperative Trajectory Optimization for Communication Coverage: An MADRL Approach

Joint AoI-Aware UAVs Trajectory Planning and Data Collection in UAV-Based IoT Systems: A Deep Reinforcement Learning Approach