Abstract:Unmanned aerial vehicles (UAVs) have the potential in delivering Internet-of-Things (IoT) services from a great height, creating an airborne domain of the IoT. In this article, we address the problem of autonomous UAV navigation in large-scale complex environments by formulating it as a Markov decision process with sparse rewards and propose an algorithm named deep reinforcement learning (RL) with nonexpert helpers (LwH). In contrast to prior RL-based methods that put huge efforts into reward shaping, we adopt the sparse reward scheme, i.e., a UAV will be rewarded if and only if it completes navigation tasks. Using the sparse reward scheme ensures that the solution is not biased toward potentially suboptimal directions. However, having no intermediate rewards hinders the agent from efficient learning since informative states are rarely encountered. To handle the challenge, we assume that a prior policy (nonexpert helper) that might be of poor performance is available to the learning agent. The prior policy plays the role of guiding the agent in exploring the state space by reshaping the behavior policy used for environmental interaction. It also assists the agent in achieving goals by setting dynamic learning objectives with increasing difficulty. To evaluate our proposed method, we construct a simulator for UAV navigation in large-scale complex environments and compare our algorithm with several baselines. Experimental results demonstrate that LwH significantly outperforms the state-of-the-art algorithms handling sparse rewards and yields impressive navigation policies comparable to those learned in the environment with dense rewards.

A Hierarchical Reinforcement Learning Algorithm Based on Attention Mechanism for UAV Autonomous Navigation

Autonomous Navigation of the UAV through Deep Reinforcement Learning with Sensor Perception Enhancement

Autonomous UAV Navigation with Adaptive Control Based on Deep Reinforcement Learning

Autonomous obstacle avoidance of UAV based on deep reinforcement learning

UAV Obstacle Avoidance by Human-in-the-Loop Reinforcement in Arbitrary 3D Environment

Autonomous Navigation of UAVs in Large-Scale Complex Environments: A Deep Reinforcement Learning Approach

Deep-Reinforcement-Learning-Based Autonomous UAV Navigation With Sparse Rewards

Explainable Deep Reinforcement Learning for UAV Autonomous Navigation

Application of Deep Reinforcement Learning in UAVs: A Review

DRAL: Deep Reinforcement Adaptive Learning for Multi-UAVs Navigation in Unknown Indoor Environment

Hierarchical Reinforcement Learning Framework towards Multi-agent Navigation

Deep-reinforcement-learning-based UAV autonomous navigation and collision avoidance in unknown environments

Monocular vision guided deep reinforcement learning UAV systems with representation learning perception

Deep-learning based autonomous-exploration for UAV navigation

Vision-Based Deep Reinforcement Learning of UAV Autonomous Navigation Using Privileged Information

A Vision Based Deep Reinforcement Learning Algorithm for UAV Obstacle Avoidance

Autonomous Navigation of UAV in Large-Scale Unknown Complex Environment with Deep Reinforcement Learning.

UAV Intelligent Coverage Navigation Based on DRL in Complex Geometrical Environments

Deep Reinforcement Learning with Heuristic Corrections for UGV Navigation

Research on Inertial Navigation Technology of Unmanned Aerial Vehicles with Integrated Reinforcement Learning Algorithm

Deep Reinforcement Learning-Based End-to-End Control for UAV Dynamic Target Tracking