Abstract:As an aerial base station, unmanned aerial vehicle (UAV) has been considered as a promising technology to assist future wireless communications due to its flexible, swift and low cost features, where resource allocation is the basis for ensuring energy-efficient UAV-assisted networks. This paper formulates a joint optimization problem of user association, UAV trajectory design and power control to maximize the channel capacity among all ground users at a limited power level in a downlink transmission. To tackle the mixed-integer non-linear programming problem, this paper proposes a clustering-aided reinforcement learning approach consisting of three consecutive stages. Firstly, modified expectation-maximization unsupervised learning algorithm is investigated to cluster the ground users, which reduces the dimensions and hence, the association complexity is reduced as well. Then, Kuhn-Munkres algorithm is incorporated for user association, which associates a UAV with the ground users via matching to the cluster, and assigns the UAVs to the centroid of the matching cluster for pre-placement, with the aim of speeding up the convergence of the following deep reinforcement learning algorithm. Finally, a multi-agent twin delayed deep deterministic (MATD3) policy gradient is proposed to solve the non-convex sub-problem, which determines the transmit power and designs the fine-tuned trajectory of UAVs. By incorporating low-bias value estimation technique, the reward of the proposed MATD3 algorithm is improved. Simulation results have demonstrated that our proposed approach achieves higher reward as well as converging faster than existing reinforcement algorithms. Besides, the clustering-aided reinforcement learning has lower computational complexity than the benchmark schemes.

Multi-objective Placement of Unmanned Aerial Vehicles As Communication Relays Based on Clustering Method

Joint Resource Allocation and Trajectory Optimization for UAV-Aided Relay Networks

Low-Complexity Joint Resource Allocation and Trajectory Design for UAV-Aided Relay Networks With the Segmented Ray-Tracing Channel Model

Resource Allocation for Multi-Cluster NOMA-UAV Networks

Joint Uplink and Downlink NOMA for UAV Relaying Network with Multi-Pair Users

A multi-UAV assisted non-orthogonal multiple access based relay system for minimal average receiving rate maximization

UAV relay network deployment through the area with barriers

Multi-UAV Relay Connectivity Optimization for Heterogeneous Users Based on Load Balancing and Throughput Maximization

Efficient On-Demand UAV Deployment and Configuration for Off-Shore Relay Communications

A Spatial Clustering and Matching Game–Based Multihop Routing Algorithm for Heterogeneous UAVs

Throughput Maximization for Multi-Cluster NOMA-UAV Networks

Centralized Unmanned Aerial Vehicle Mesh Network Placement Scheme: A Multi-Objective Evolutionary Algorithm Approach

Power Allocation and Successive Relay Selection in UAV-assisted Cooperative NOMA System

Distributed Relay Selection for Heterogeneous UAV Communication Networks Using A Many-to-Many Matching Game Without Substitutability

UAV Swarm-enabled Collaborative Secure Relay Communications with Time-domain Colluding Eavesdropper

Multi-Agent Learning Based Packet Routing in Multi-Hop UAV Relay Network

Adaptive Resource Allocation for Emergency Communications with Unmanned Aerial Vehicle-Assisted Free Space Optical/Radio Frequency Relay System

Resource Allocation in UAV-Assisted Networks: A Clustering-Aided Reinforcement Learning Approach

UAV Virtual Antenna Array Deployment for Uplink Interference Mitigation in Data Collection Networks

Joint 3-D Positioning and Power Allocation for UAV Relay Aided by Geographic Information

Placement of UAVs to Reconnect Lost Subnetworks in Wireless Sensor Networks.