Abstract:As an aerial base station, unmanned aerial vehicle (UAV) has been considered as a promising technology to assist future wireless communications due to its flexible, swift and low cost features, where resource allocation is the basis for ensuring energy-efficient UAV-assisted networks. This paper formulates a joint optimization problem of user association, UAV trajectory design and power control to maximize the channel capacity among all ground users at a limited power level in a downlink transmission. To tackle the mixed-integer non-linear programming problem, this paper proposes a clustering-aided reinforcement learning approach consisting of three consecutive stages. Firstly, modified expectation-maximization unsupervised learning algorithm is investigated to cluster the ground users, which reduces the dimensions and hence, the association complexity is reduced as well. Then, Kuhn-Munkres algorithm is incorporated for user association, which associates a UAV with the ground users via matching to the cluster, and assigns the UAVs to the centroid of the matching cluster for pre-placement, with the aim of speeding up the convergence of the following deep reinforcement learning algorithm. Finally, a multi-agent twin delayed deep deterministic (MATD3) policy gradient is proposed to solve the non-convex sub-problem, which determines the transmit power and designs the fine-tuned trajectory of UAVs. By incorporating low-bias value estimation technique, the reward of the proposed MATD3 algorithm is improved. Simulation results have demonstrated that our proposed approach achieves higher reward as well as converging faster than existing reinforcement algorithms. Besides, the clustering-aided reinforcement learning has lower computational complexity than the benchmark schemes.

Cooperative Deep Learning-Based Uplink Distributed Fair Resource Allocation for Aerial Reconfigurable Intelligent Surfaces Wireless Networks

Joint Deployment and Resource Management for VLC-Enabled RISs-Assisted UAV Networks

Penalized Reinforcement Learning-Based Energy-Efficient UAV-RIS Assisted Maritime Uplink Communications Against Jamming

Reconfigurable Intelligent Surface-Assisted Aerial-Terrestrial Communications Via Multi-Task Learning

Hybrid Uplink and Downlink Transmissions for Full-Duplex UAV Communication With RIS

Fairness-Driven Optimization of RIS-Augmented 5G Networks for Seamless 3D UAV Connectivity Using DRL Algorithms

Fairness Enhancement of UAV Systems with Hybrid Active-Passive RIS

Meta Reinforcement Learning for Resource Allocation in Aerial Active-RIS-assisted Networks with Rate-Splitting Multiple Access

Optimal Resource Allocation for Multi-UAV Assisted Visible Light Communication

Joint 3D trajectory and phase shift optimization via deep reinforcement learning for RIS-assisted UAV communication systems

Energy-Efficient Communication Networks via Multiple Aerial Reconfigurable Intelligent Surfaces: DRL and Optimization Approach

Resource Allocation in UAV-Assisted Networks: A Clustering-Aided Reinforcement Learning Approach

Robust Secure UAV Communications with the Aid of Reconfigurable Intelligent Surfaces

Energy Harvesting Reconfigurable Intelligent Surface for UAV Based on Robust Deep Reinforcement Learning

Blocklength Allocation and Power Control in UAV-Assisted URLLC System via Multi-agent Deep Reinforcement Learning

Reconfigurable Intelligent Surface-Assisted mmWave multi-UAV Wireless Cellular Networks

CoMP and RIS-assisted Multicast Transmission in a Multi-UAV Communication System

Reconfigurable Intelligent Surface Assisted UAV Communication: Joint Trajectory Design and Passive Beamforming

Reconfigurable Wireless Relaying With Multi-UAV-Carried Intelligent Reflecting Surfaces

Low-Resolution Optimization for an Unmanned Aerial Vehicle Communication Network under a Passive Reconfigurable Intelligent Surface and Active Reconfigurable Intelligent Surface