Abstract:As an aerial base station, unmanned aerial vehicle (UAV) has been considered as a promising technology to assist future wireless communications due to its flexible, swift and low cost features, where resource allocation is the basis for ensuring energy-efficient UAV-assisted networks. This paper formulates a joint optimization problem of user association, UAV trajectory design and power control to maximize the channel capacity among all ground users at a limited power level in a downlink transmission. To tackle the mixed-integer non-linear programming problem, this paper proposes a clustering-aided reinforcement learning approach consisting of three consecutive stages. Firstly, modified expectation-maximization unsupervised learning algorithm is investigated to cluster the ground users, which reduces the dimensions and hence, the association complexity is reduced as well. Then, Kuhn-Munkres algorithm is incorporated for user association, which associates a UAV with the ground users via matching to the cluster, and assigns the UAVs to the centroid of the matching cluster for pre-placement, with the aim of speeding up the convergence of the following deep reinforcement learning algorithm. Finally, a multi-agent twin delayed deep deterministic (MATD3) policy gradient is proposed to solve the non-convex sub-problem, which determines the transmit power and designs the fine-tuned trajectory of UAVs. By incorporating low-bias value estimation technique, the reward of the proposed MATD3 algorithm is improved. Simulation results have demonstrated that our proposed approach achieves higher reward as well as converging faster than existing reinforcement algorithms. Besides, the clustering-aided reinforcement learning has lower computational complexity than the benchmark schemes.

Joint UAV Deployment and Resource Allocation: a Personalized Federated Deep Reinforcement Learning Approach

Joint Multi-UAV Deployment and Resource Allocation Based on Personalized Federated Deep Reinforcement Learning.

Joint Optimization of Multi-UAV Deployment and User Association Via Deep Reinforcement Learning for Long-Term Communication Coverage

UAV-Enabled Energy-Efficient Aerial Computing: A Federated Deep Reinforcement Learning Approach

Resource Allocation in UAV-Assisted Networks: A Clustering-Aided Reinforcement Learning Approach

Resource Allocation in UAV-D2D Networks: A Scalable Heterogeneous Multi-Agent Deep Reinforcement Learning Approach

Joint Power Allocation and 3D Deployment for UAV-BSs: A Game Theory Based Deep Reinforcement Learning Approach

Blocklength Allocation and Power Control in UAV-Assisted URLLC System via Multi-agent Deep Reinforcement Learning

Deep Reinforcement Learning-enabled Dynamic UAV Deployment and Power Control in Multi-UAV Wireless Networks

A Multi-Agent Collaborative Environment Learning Method for UAV Deployment and Resource Allocation

Joint 3D trajectory and phase shift optimization via deep reinforcement learning for RIS-assisted UAV communication systems

Mean Field Deep Reinforcement Learning for Fair and Efficient UAV Control

Federated deep reinforcement learning based trajectory design for UAV-assisted networks with mobile ground devices

Deep Reinforcement Learning for Aerial Data Collection in Hybrid-Powered NOMA-IoT Networks

Joint AP Selection and Task Offloading Based on Deep Reinforcement Learning for Urban-Micro Cell-Free UAV Network

Deep Reinforcement Learning Based Resource Allocation and Trajectory Planning in Integrated Sensing and Communications UAV Network

Multi-UAV Hierarchical Intelligent Traffic Offloading Network Optimization Based on Deep Federated Learning

UAV-assisted fair communications for multi-pair users: A multi-agent deep reinforcement learning method

Deep Reinforcement Learning-based UAV Control for Optimized Energy Efficiency and Throughput in UAV-assisted Communication

Learning-based User Association and Dynamic Resource Allocation in Multi-Connectivity Enabled Unmanned Aerial Vehicle Networks

Energy-Efficient UAV Control for Effective and Fair Communication Coverage: A Deep Reinforcement Learning Approach