Abstract:As an aerial base station, unmanned aerial vehicle (UAV) has been considered as a promising technology to assist future wireless communications due to its flexible, swift and low cost features, where resource allocation is the basis for ensuring energy-efficient UAV-assisted networks. This paper formulates a joint optimization problem of user association, UAV trajectory design and power control to maximize the channel capacity among all ground users at a limited power level in a downlink transmission. To tackle the mixed-integer non-linear programming problem, this paper proposes a clustering-aided reinforcement learning approach consisting of three consecutive stages. Firstly, modified expectation-maximization unsupervised learning algorithm is investigated to cluster the ground users, which reduces the dimensions and hence, the association complexity is reduced as well. Then, Kuhn-Munkres algorithm is incorporated for user association, which associates a UAV with the ground users via matching to the cluster, and assigns the UAVs to the centroid of the matching cluster for pre-placement, with the aim of speeding up the convergence of the following deep reinforcement learning algorithm. Finally, a multi-agent twin delayed deep deterministic (MATD3) policy gradient is proposed to solve the non-convex sub-problem, which determines the transmit power and designs the fine-tuned trajectory of UAVs. By incorporating low-bias value estimation technique, the reward of the proposed MATD3 algorithm is improved. Simulation results have demonstrated that our proposed approach achieves higher reward as well as converging faster than existing reinforcement algorithms. Besides, the clustering-aided reinforcement learning has lower computational complexity than the benchmark schemes.

Reinforcement Learning Based Scheduling for Heterogeneous UAV Networking.

Deep Reinforcement Learning for Scheduling and Offloading in UAV-Assisted Mobile Edge Networks

UAV-Assisted 5G/6G Networks: Joint Scheduling and Resource Allocation Based on Asynchronous Reinforcement Learning

Transfer Learning Empowered Resource Scheduling in Aerial Edge Networks.

Enabling Efficient Scheduling in Large-Scale UAV-Assisted Mobile-Edge Computing via Hierarchical Reinforcement Learning

Reinforcement Learning-Based Computing and Transmission Scheduling for LTE-U-Enabled IoT

Joint UAV trajectory and communication design with heterogeneous multi-agent reinforcement learning

Scheduling UAV Swarm with Attention-based Graph Reinforcement Learning for Ground-to-air Heterogeneous Data Communication

Deep Reinforcement Learning Based Placement for Integrated Access Backhauling in UAV-Assisted Wireless Networks

Joint AP Selection and Task Offloading Based on Deep Reinforcement Learning for Urban-Micro Cell-Free UAV Network

Deep Reinforcement Learning for Joint Trajectory Planning, Transmission Scheduling, and Access Control in UAV-Assisted Wireless Sensor Networks

Hierarchical Network Slicing for UAV-assisted Wireless Networks with Deployment Optimization

A Reinforcement Learning Based User Association Algorithm for UAV Networks

HAPS-UAV-Enabled Heterogeneous Networks: A Deep Reinforcement Learning Approach

Priority-aware path planning and user scheduling for UAV-mounted MEC networks: A deep reinforcement learning approach

Reinforcement Learning Placement Algorithm for Optimization of UAV Network in Wireless Communication

Blocklength Allocation and Power Control in UAV-Assisted URLLC System via Multi-agent Deep Reinforcement Learning

On-Demand Multiplexing of eMBB/URLLC Traffic in a Multi-UAV Relay Network

Throughput Optimization in Heterogeneous Swarms of Unmanned Aircraft Systems for Advanced Aerial Mobility

Resource Scheduling Based on Deep Reinforcement Learning in UAV Assisted Emergency Communication Networks

Resource Allocation in UAV-Assisted Networks: A Clustering-Aided Reinforcement Learning Approach