Abstract:Despite being a widely adopted development framework for unmanned aerial vehicle (UAV), deep reinforcement learning (DRL) is often considered sample inefficient. Particularly, UAV struggles to fully explore the state and action space in environments with sparse rewards. While some exploration algorithms have been proposed to overcome the challenge of sparse rewards, they are not specifically tailored for UAV platform. Consequently, applying those algorithms to UAV path planning may lead to problems such as unstable training processes and neglect of action space comprehension, possibly causing negative impacts on the path planning results. To address the problem of sparse rewards in UAV path planning, we propose an information-theoretic exploration algorithm, Entropy Explorer (EE), specifically for UAV platform. The proposed EE generates intrinsic rewards based on state entropy and action entropy to compensate for the scarcity of extrinsic rewards. To further improve sampling efficiency, a framework integrating EE and Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithms is proposed. Finally, the TD3-EE algorithm is tested in Airsim and compared against benchmarking algorithms. The simulation outcomes manifest that TD3-EE effectively stimulates the UAV to comprehensively explore both state and action spaces, thereby attaining superior performance compared to the benchmark algorithms in the realm of path planning.

UAV Coverage Path Planning Based on Deep Reinforcement Learning

A UAV Coverage Path Planning Algorithm Based on Double Deep Q-Network

Path Planning for UAV Ground Target Tracking via Deep Reinforcement Learning

Dynamic Scene Path Planning of UAVs Based on Deep Reinforcement Learning

Towards Real-Time Path Planning through Deep Reinforcement Learning for a UAV in Dynamic Environments

Adaptive speed planning for Unmanned Vehicle Based on Deep Reinforcement Learning

Collaborative Coverage Path Planning of UAV Cluster based on Deep Reinforcement Learning

Path Planning Research of a UAV Base Station Searching for Disaster Victims' Location Information Based on Deep Reinforcement Learning

Enhancing Offline Coverage Path Planning with Deep Reinforcement Learning

Path Planning of UAV Base Station Based on Deep Reinforcement Learning

Deep Reinforcement Learning-based Collaborative Multi-UAV Coverage Path Planning

A Motion Camouflage-Inspired Path Planning Method for UAVs Based on Reinforcement Learning

ARDDQN: Attention Recurrent Double Deep Q-Network for UAV Coverage Path Planning and Data Harvesting

An Algorithm of Complete Coverage Path Planning for Unmanned Surface Vehicle Based on Reinforcement Learning

UAV Path Planning for Wireless Data Harvesting: A Deep Reinforcement Learning Approach

Learning-Based UAV Coverage-Aware Path Planning in Large-scale Urban Environments

Improve Exploration in Deep Reinforcement Learning for UAV Path Planning using State and Action Entropy

Deep Reinforcement Learning for UAV Intelligent Mission Planning

Deep Reinforcement Learning-Driven UAV Data Collection Path Planning: A Study on Minimizing AoI

Path Planning of Unmanned Aerial Vehicle in Complex Environments Based on State-Detection Twin Delayed Deep Deterministic Policy Gradient

B-APFDQN: A UAV Path Planning Algorithm Based on Deep Q-Network and Artificial Potential Field