Green Communications: RIS-Assisted Fixed-Wing UAV Coverage Scheme Based on Deep Reinforcement Learning
Na Lin,Chunxiao Liu,Tianxiong Wu,Ammar Hawbani,Liang Zhao,Shaohua Wan,Mohsen Guizani
DOI: https://doi.org/10.1109/jiot.2024.3483778
IF: 10.6
2024-01-01
IEEE Internet of Things Journal
Abstract:Recently, fixed-wing unmanned aerial vehicles (fixed-wing UAVs) are able to extend the communications mission time and ease of deployment due to their powerful onboard capabilities and flexibility, and reflective intelligent surfaces (RIS) are capable of reflecting links to avoid obstacles and thus improve channel gain. Therefore, RIS-assisted fixed-wing UAVs are widely used in wireless communications. Nevertheless, fixed-wing UAVs have limited energy, so improving energy efficiency is critical. This paper focuses on energy efficiency optimization problems under RIS-assisted fixed-wing UAV communications. In communications coverage systems, the flight trajectory of fixed-wing UAVs and service scheduling to ground nodes (GNs) significantly impact energy efficiency. Existing work often adopts circular trajectory and traditional deep reinforcement learning (DRL) algorithms for optimizing trajectory and service scheduling. However, circular trajectory can not adapted to the GN distribution well. In addition, the traditional DRL algorithm has two drawbacks: the efficiency of exploring the empirical process is low, and the accuracy of handling the hybrid action space needs to be higher. Thus, we propose the midpoint iteration convex hull (MICH) algorithm based on the Graham scan to design trajectories that can be adapted to the distribution of the GNs. In addition, we propose the action screening virtual and real experience (AS-VRE) mechanism and the N-steps hybrid deep q and policy network (NsHQPN) algorithm to address the low exploration efficiency and the low fetch accuracy in handling the hybrid action space. Experiments show that our proposed MICH algorithm, AS-VRE mechanism, and NsHQPN algorithm can effectively improve the system energy efficiency and outperform other baseline schemes.