Abstract:Unmanned aerial vehicle (UAV) can be deployed as aerial base station to provide communication services for the user equipments (UEs). However, in urban environments, the links between UAV and UEs might be frequently blocked by obstacles, leading to severely adverse effects on the quality of service (QoS) of UEs. Additionally, due to the limited energy of the UAV, it might not always be feasible to re-establish the line-of-sight (LoS) links by frequently adjusting the positions of the UAV. In this context, the reconfigurable intelligent surface (RIS) is utilized to enhance the transmission range of UAV-UE links by reflecting incident signals to UEs. In this paper, we investigate the RIS-assisted UAV communication systems with the goal of maximizing the energy efficiency of the UAV through a joint optimization of the UAV's trajectory and the RIS's phase shift. The formulated optimization problem is non-convex, and challenging to solve in a polynomial time. Therefore, an effective deep reinforcement learning (DRL)-based solution, named Dueling DQN-PER is proposed, which combines the Dueling DQN algorithm with the prioritized experience replay (PER) technique. To ensure the fairness among all UEs, we design a service fairness index, and integrate it into the reward function when designing the proposed algorithm. Numerical results demonstrate that: 1) the proposed Dueling DQN-PER algorithm is capable of improving the system energy efficiency and has a better training performance than benchmark schemes; 2) by devising the service fairness index, the fairness among all UEs is ensured while enhancing the system performance in energy efficiency; 3) the RIS-assisted UAV communication systems benefit from significant energy efficiency gain over the systems without RIS.

Joint 3D Deployment and Power Allocation for UAV-BS: A Deep Reinforcement Learning Approach

An Actor-Critic-Based UAV-BSs Deployment Method for Dynamic Environments

Towards 3D Deployment of UAV Base Stations in Uneven Terrain.

RSS-Based UAV-BS 3-D Mobility Management via Policy Gradient Deep Reinforcement Learning

Joint 3D Deployment and Resource Allocation for UAV-assisted Integrated Communication and Localization

Multi-objective Deep Reinforcement Learning Based Joint Beamforming and Power Allocation in UAV Assisted Cellular Communication

Deep Reinforcement Learning Based 3D UAV Trajectory Design and Frequency Band Allocation.

Improved Genetic Algorithm Based 3-D Deployment of UAVs

3D UAV Trajectory Design and Frequency Band Allocation for Energy-Efficient and Fair Communication: A Deep Reinforcement Learning Approach

3D-Trajectory and Phase-Shift Design for RIS-Assisted UAV Systems Using Deep Reinforcement Learning

Dynamic environment UAV deployment algorithm based on potential game theory

Three-Dimensional Trajectory and Resource Allocation Optimization in Multi-Unmanned Aerial Vehicle Multicast System: A Multi-Agent Reinforcement Learning Method

UAV-Assisted Enhanced Coverage and Capacity in Dynamic MU-mMIMO IoT Systems: A Deep Reinforcement Learning Approach

Intelligent flying-beamformer for hybrid mmWave systems: A deep reinforcement learning approach

Three-dimensional deep reinforcement learning for trajectory and resource optimization in UAV communication systems

Placement Optimization of Aerial Base Stations with Deep Reinforcement Learning

Joint 3D trajectory and phase shift optimization via deep reinforcement learning for RIS-assisted UAV communication systems

A Novel Joint DRL-Based Utility Optimization for UAV Data Services

Reinforcement Learning in Multiple-UAV Networks: Deployment and Movement Design

Three-Dimension Trajectory Design for Multi-UAV Wireless Network With Deep Reinforcement Learning

Deep Q-Network Based Resource Allocation for UAV-assisted Ultra-Dense Networks