Abstract:Unmanned aerial vehicle (UAV) can be deployed as aerial base station to provide communication services for the user equipments (UEs). However, in urban environments, the links between UAV and UEs might be frequently blocked by obstacles, leading to severely adverse effects on the quality of service (QoS) of UEs. Additionally, due to the limited energy of the UAV, it might not always be feasible to re-establish the line-of-sight (LoS) links by frequently adjusting the positions of the UAV. In this context, the reconfigurable intelligent surface (RIS) is utilized to enhance the transmission range of UAV-UE links by reflecting incident signals to UEs. In this paper, we investigate the RIS-assisted UAV communication systems with the goal of maximizing the energy efficiency of the UAV through a joint optimization of the UAV's trajectory and the RIS's phase shift. The formulated optimization problem is non-convex, and challenging to solve in a polynomial time. Therefore, an effective deep reinforcement learning (DRL)-based solution, named Dueling DQN-PER is proposed, which combines the Dueling DQN algorithm with the prioritized experience replay (PER) technique. To ensure the fairness among all UEs, we design a service fairness index, and integrate it into the reward function when designing the proposed algorithm. Numerical results demonstrate that: 1) the proposed Dueling DQN-PER algorithm is capable of improving the system energy efficiency and has a better training performance than benchmark schemes; 2) by devising the service fairness index, the fairness among all UEs is ensured while enhancing the system performance in energy efficiency; 3) the RIS-assisted UAV communication systems benefit from significant energy efficiency gain over the systems without RIS.

Optimal Design for Trajectory and Phase-Shift in UAV-Mounted-RIS Communications with Reinforcement Learning

3D-Trajectory and Phase-Shift Design for RIS-Assisted UAV Systems Using Deep Reinforcement Learning

Joint 3D trajectory and phase shift optimization via deep reinforcement learning for RIS-assisted UAV communication systems

Deep Reinforcement Learning and NOMA-Based Multi-Objective RIS-Assisted IS-UAV-TNs: Trajectory Optimization and Beamforming Design

Joint Trajectory and Passive Beamforming Design for Secure UAV Networks with RIS

Three-Dimensional Trajectory Design for Multi-User MISO UAV Communications: A Deep Reinforcement Learning Approach

Capacity Maximization in RIS-UAV Networks: A DDQN-Based Trajectory and Phase Shift Optimization Approach

Three-dimensional deep reinforcement learning for trajectory and resource optimization in UAV communication systems

Penalized Reinforcement Learning-Based Energy-Efficient UAV-RIS Assisted Maritime Uplink Communications Against Jamming

Reconfigurable Intelligent Surface Assisted UAV Communication: Joint Trajectory Design and Passive Beamforming

Reconfigurable Intelligent Surface-Assisted Aerial-Terrestrial Communications Via Multi-Task Learning

Joint Trajectory and Passive Beamforming Design for Intelligent Reflecting Surface-Aided UAV Communications: A Deep Reinforcement Learning Approach

RIS-Assisted Jamming Rejection and Path Planning for UAV-Borne IoT Platform: A New Deep Reinforcement Learning Framework

Joint Deployment and Resource Management for VLC-Enabled RISs-Assisted UAV Networks

Deep Reinforcement Learning-Driven Reconfigurable Intelligent Surface-Assisted Radio Surveillance with a Fixed-Wing UAV.

Collaborative Reinforcement Learning Based Unmanned Aerial Vehicle (UAV) Trajectory Design for 3D UAV Tracking

Hybrid Uplink and Downlink Transmissions for Full-Duplex UAV Communication With RIS

Energy-Efficient Symbiotic UAV-Enabled MEC Networks via RIS: Joint Trajectory and Phase-Shift Control Optimization

Reconfigurable Intelligent Surface Assisted Multiuser MISO Systems Exploiting Deep Reinforcement Learning

Deep Reinforcement Learning for Trajectory and Phase Shift Optimization of Aerial RIS in CoMP-NOMA Networks