Abstract:Cellular-connected unmanned aerial vehicle (UAV) is a promising technology to unlock the full potential of UAVs in the future by reusing the cellular base stations (BSs) to enable their air-ground communications. However, how to achieve ubiquitous three-dimensional (3D) communication coverage for the UAVs in the sky is a new challenge. In this paper, we tackle this challenge by a new coverage-aware navigation approach, which exploits the UAV's controllable mobility to design its navigation/trajectory to avoid the cellular BSs' coverage holes while accomplishing their missions. To this end, we formulate an UAV trajectory optimization problem to minimize the weighted sum of its mission completion time and expected communication outage duration, which, however, cannot be solved by the standard optimization techniques due to the lack of an accurate and tractable end-to-end communication model in practice. To overcome this difficulty, we propose a new solution approach based on the technique of deep reinforcement learning (DRL). Specifically, by leveraging the state-of-the-art dueling double deep Q network (dueling DDQN) with multi-step learning, we first propose a UAV navigation algorithm based on direct RL, where the signal measurement at the UAV is used to directly train the action-value function of the navigation policy. To further improve the performance, we propose a new framework called simultaneous navigation and radio mapping (SNARM), where the UAV's signal measurement is used not only for training the DQN directly, but also to create a radio map that is able to predict the outage probabilities at all locations in the area of interest. This enables the generation of simulated UAV trajectories and predicting their expected returns, which are then used to further train the DQN via Dyna technique, thus great-y improving the learning efficiency.

Deep-learning based autonomous-exploration for UAV navigation

Autonomous UAV Navigation with Adaptive Control Based on Deep Reinforcement Learning

Autonomous Navigation of UAVs in Large-Scale Complex Environments: A Deep Reinforcement Learning Approach

DRAL: Deep Reinforcement Adaptive Learning for Multi-UAVs Navigation in Unknown Indoor Environment

UAV Autonomous Navigation Based on Deep Reinforcement Learning in Highly Dynamic and High-Density Environments

Autonomous Navigation of UAV in Large-Scale Unknown Complex Environment with Deep Reinforcement Learning.

Autonomous Navigation of the UAV through Deep Reinforcement Learning with Sensor Perception Enhancement

Autonomous Navigation for Cellular-Connected UAV in Highly Dynamic Environments: A Deep Reinforcement Learning Approach

Learning to Explore Indoor Environments using Autonomous Micro Aerial Vehicles

Deep-reinforcement-learning-based UAV autonomous navigation and collision avoidance in unknown environments

Deep-Reinforcement-Learning-Based Autonomous UAV Navigation With Sparse Rewards

A Vision Based Deep Reinforcement Learning Algorithm for UAV Obstacle Avoidance

A Motion Camouflage-Inspired Path Planning Method for UAVs Based on Reinforcement Learning

UAV Obstacle Avoidance by Human-in-the-Loop Reinforcement in Arbitrary 3D Environment

Autonomous Exploration Method for Fast Unknown Environment Mapping by Using UAV Equipped with Limited FOV Sensor

Advancements in UAV Path Planning: A Deep Reinforcement Learning Approach with Soft Actor-Critic for Enhanced Navigation

Safe Navigation for UAV-Enabled Data Dissemination by Deep Reinforcement Learning in Unknown Environments

Improve Exploration in Deep Reinforcement Learning for UAV Path Planning using State and Action Entropy

UAV Intelligent Coverage Navigation Based on DRL in Complex Geometrical Environments

Simultaneous Navigation and Radio Mapping for Cellular-Connected UAV With Deep Reinforcement Learning

Autonomous Navigation of Unmanned Vehicle Through Deep Reinforcement Learning