Abstract:Cellular-connected unmanned aerial vehicle (UAV) is a promising technology to unlock the full potential of UAVs in the future by reusing the cellular base stations (BSs) to enable their air-ground communications. However, how to achieve ubiquitous three-dimensional (3D) communication coverage for the UAVs in the sky is a new challenge. In this paper, we tackle this challenge by a new coverage-aware navigation approach, which exploits the UAV's controllable mobility to design its navigation/trajectory to avoid the cellular BSs' coverage holes while accomplishing their missions. To this end, we formulate an UAV trajectory optimization problem to minimize the weighted sum of its mission completion time and expected communication outage duration, which, however, cannot be solved by the standard optimization techniques due to the lack of an accurate and tractable end-to-end communication model in practice. To overcome this difficulty, we propose a new solution approach based on the technique of deep reinforcement learning (DRL). Specifically, by leveraging the state-of-the-art dueling double deep Q network (dueling DDQN) with multi-step learning, we first propose a UAV navigation algorithm based on direct RL, where the signal measurement at the UAV is used to directly train the action-value function of the navigation policy. To further improve the performance, we propose a new framework called simultaneous navigation and radio mapping (SNARM), where the UAV's signal measurement is used not only for training the DQN directly, but also to create a radio map that is able to predict the outage probabilities at all locations in the area of interest. This enables the generation of simulated UAV trajectories and predicting their expected returns, which are then used to further train the DQN via Dyna technique, thus great-y improving the learning efficiency.

Autonomous Obstacle Avoidance and Communication Capacity Optimization for UAV-Assisted VLC Systems

Trajectory Planning for Cross-Interface UAV-Assisted VLC Systems

Autonomous obstacle avoidance of UAV based on deep reinforcement learning

Automatic Trajectory Planning for UAV Aided Visible Light Communication Systems

Joint Optimization of Multi-UAV Deployment and User Association Via Deep Reinforcement Learning for Long-Term Communication Coverage

Deep Reinforcement Learning-based UAV Control for Optimized Energy Efficiency and Throughput in UAV-assisted Communication

Joint 3D trajectory and phase shift optimization via deep reinforcement learning for RIS-assisted UAV communication systems

UAV Trajectory and Resource Optimization for NOMA-VLC Systems Via HA-DRL Algorithm

Interpretable and Secure Trajectory Optimization for UAV-Assisted Communication

Energy-Efficient UAV Control for Effective and Fair Communication Coverage: A Deep Reinforcement Learning Approach

Deep Reinforcement Learning Based Trajectory Design and Resource Allocation for UAV-Assisted Communications

Multi-UAV Dynamic Wireless Networking with Deep Reinforcement Learning

Dynamic Channel Allocation for Multi-UAVs: A Deep Reinforcement Learning Approach

A deep reinforcement learning framework and its implementation for UAV-aided covert communication

Deep Reinforcement Learning Based Resource Allocation and Trajectory Planning in Integrated Sensing and Communications UAV Network

Blocklength Allocation and Power Control in UAV-Assisted URLLC System via Multi-agent Deep Reinforcement Learning

A Vision Based Deep Reinforcement Learning Algorithm for UAV Obstacle Avoidance

A Novel Joint DRL-Based Utility Optimization for UAV Data Services

Deep Reinforcement Learning for Channel and Power Allocation in UAV-enabled IoT Systems

Simultaneous Navigation and Radio Mapping for Cellular-Connected UAV With Deep Reinforcement Learning

Deep Q-Network Based Dynamic Trajectory Design for UAV-Aided Emergency Communications