Abstract:As an aerial base station, unmanned aerial vehicle (UAV) has been considered as a promising technology to assist future wireless communications due to its flexible, swift and low cost features, where resource allocation is the basis for ensuring energy-efficient UAV-assisted networks. This paper formulates a joint optimization problem of user association, UAV trajectory design and power control to maximize the channel capacity among all ground users at a limited power level in a downlink transmission. To tackle the mixed-integer non-linear programming problem, this paper proposes a clustering-aided reinforcement learning approach consisting of three consecutive stages. Firstly, modified expectation-maximization unsupervised learning algorithm is investigated to cluster the ground users, which reduces the dimensions and hence, the association complexity is reduced as well. Then, Kuhn-Munkres algorithm is incorporated for user association, which associates a UAV with the ground users via matching to the cluster, and assigns the UAVs to the centroid of the matching cluster for pre-placement, with the aim of speeding up the convergence of the following deep reinforcement learning algorithm. Finally, a multi-agent twin delayed deep deterministic (MATD3) policy gradient is proposed to solve the non-convex sub-problem, which determines the transmit power and designs the fine-tuned trajectory of UAVs. By incorporating low-bias value estimation technique, the reward of the proposed MATD3 algorithm is improved. Simulation results have demonstrated that our proposed approach achieves higher reward as well as converging faster than existing reinforcement algorithms. Besides, the clustering-aided reinforcement learning has lower computational complexity than the benchmark schemes.

Matrix Factorization and Deep Autoencoder based Clustering Scheme for Large-scale UAV Networks

An Energy Balance and Mobility Prediction Clustering Algrithm for Large-Scale UAV Ad Hoc Networks

Comprehensive Mobility Prediction Based Clustering Algorithm for Ad Hoc UAV Networks

Clustering Strategy of UAV Network Based on Deep Q-learning

ClusterFusion: Real-time Relative Positioning and Dense Reconstruction for UAV Cluster

A Two-Layer Task Assignment Algorithm for UAV Swarm Based on Feature Weight Clustering

Resource Allocation in UAV-Assisted Networks: A Clustering-Aided Reinforcement Learning Approach

Placement of Unmanned Aerial Vehicles As Communication Relays in Two-Tiered Multi-Agent System: Clustering Based Methods

Bio-Inspired Mobility Prediction Clustering Algorithm for Ad Hoc UAV Networks

Adaptive Enhanced Weighted Clustering Algorithm for UAV Swarm

Deep neural network-based clustering algorithm for multiple flying reconfigurable intelligent surfaces-supported bulk systems

Fission Spectral Clustering Strategy For UAV Swarm Networks

Research on Multi-UAV Dynamic Mission Assignment Method Based on Clustering Algorithm

Multidimensional Scaling Algorithm for Mobile Swarming UAVs Localization

Efficient Data Collection in Large-Scale UAV-aided Wireless Sensor Networks

A Dual-Cluster-Head Based Medium Access Control for Large-Scale UAV Ad-Hoc Networks

Joint Clustering and 3-D UAV Deployment for Delay-Aware UAV-Enabled MTC Data Collection Networks

Data Collection In UAV-Assisted Wireless IoT Networks: An Improved Clustering Algorithm and Resource Allocation Design

On UAV Serving Node Deployment for Temporary Coverage in Forest Environment: A Hierarchical Deep Reinforcement Learning Approach

UAV-Enabled Spatial Data Sampling in Large-Scale IoT Systems Using Denoising Autoencoder Neural Network

Cooperative Networking Strategy of UAV Cluster for Large-Scale WSNs