Abstract:As an aerial base station, unmanned aerial vehicle (UAV) has been considered as a promising technology to assist future wireless communications due to its flexible, swift and low cost features, where resource allocation is the basis for ensuring energy-efficient UAV-assisted networks. This paper formulates a joint optimization problem of user association, UAV trajectory design and power control to maximize the channel capacity among all ground users at a limited power level in a downlink transmission. To tackle the mixed-integer non-linear programming problem, this paper proposes a clustering-aided reinforcement learning approach consisting of three consecutive stages. Firstly, modified expectation-maximization unsupervised learning algorithm is investigated to cluster the ground users, which reduces the dimensions and hence, the association complexity is reduced as well. Then, Kuhn-Munkres algorithm is incorporated for user association, which associates a UAV with the ground users via matching to the cluster, and assigns the UAVs to the centroid of the matching cluster for pre-placement, with the aim of speeding up the convergence of the following deep reinforcement learning algorithm. Finally, a multi-agent twin delayed deep deterministic (MATD3) policy gradient is proposed to solve the non-convex sub-problem, which determines the transmit power and designs the fine-tuned trajectory of UAVs. By incorporating low-bias value estimation technique, the reward of the proposed MATD3 algorithm is improved. Simulation results have demonstrated that our proposed approach achieves higher reward as well as converging faster than existing reinforcement algorithms. Besides, the clustering-aided reinforcement learning has lower computational complexity than the benchmark schemes.

Research on Clustering Optimization Algorithm for UAV Cluster Network

An Energy Balance and Mobility Prediction Clustering Algrithm for Large-Scale UAV Ad Hoc Networks

The Clustering Algorithm of Uav Networking in Near-Space

Communication and Trajectory Design in UAV-enabled Flying Network.

Bio-Inspired Mobility Prediction Clustering Algorithm for Ad Hoc UAV Networks

Comprehensive Mobility Prediction Based Clustering Algorithm for Ad Hoc UAV Networks

Topology optimization algorithm for UAV formation based on wireless ultraviolet communication

Clustering Strategy of UAV Network Based on Deep Q-learning

Formation Flying and Obstacle Avoidance Control of UAV Cluster Based on Backbone Network

Coverage Capacity Optimization of UAV-enabled Networks

UAV Communication Network Modeling and Energy Consumption Optimization Based on Routing Algorithm

Placement of Unmanned Aerial Vehicles As Communication Relays in Two-Tiered Multi-Agent System: Clustering Based Methods

Path Planning of UAVs Based on Improved Clustering Algorithm and Ant Colony System Algorithm

A Clustering Scheme Based on the Binary Whale Optimization Algorithm in FANET

A Dynamic Cluster Head Selecting Algorithm for UAV Ad Hoc Networks

Resource Allocation in UAV-Assisted Networks: A Clustering-Aided Reinforcement Learning Approach

Novel Hybrid Deep Optimized Machine Learning Model for Energy Efficient Cluster Selection in Unmanned Aerial Vehicle Networks

Trajectory Optimization Algorithm of UAV Based on Pointer Network

Joint Trajectory and Energy Efficiency Optimization for Multi-UAV Assisted Offloading

Topology Control Algorithm for UAV Ad Hoc Networks

Research on Deterministic Transmission Services Scheduling in UAV Cluster Network