Autonomous UAV Navigation with Adaptive Control Based on Deep Reinforcement Learning

Yongfeng Yin,Zhetao Wang,Lili Zheng,Qingran Su,Yang Guo

DOI: https://doi.org/10.3390/electronics13132432

IF: 2.9

2024-06-22

Electronics

Abstract:Unmanned aerial vehicle (UAV) navigation plays a crucial role in its ability to perform autonomous missions in complex environments. Most of the existing reinforcement learning methods to solve the UAV navigation problem fix the flight altitude and velocity, which largely reduces the difficulty of the algorithm. But the methods without adaptive control are not suitable in low-altitude environments with complex situations, generally suffering from weak obstacle avoidance. Some UAV navigation studies with adaptive flight only have weak obstacle avoidance capabilities. To address the problem of UAV navigation in low-altitude environments, we construct autonomous UAV navigation in 3D environments with adaptive control as a Markov decision process and propose a deep reinforcement learning algorithm. To solve the problem of weak obstacle avoidance, we creatively propose the guide attention method to make a UAV's decision focus shift between the navigation task and obstacle avoidance task according to changes in the obstacle. We raise a novel velocity-constrained loss function and add it to the original actor loss to improve the UAV's velocity control capability. Simulation experiment results demonstrate that our algorithm outperforms some of the state-of-the-art deep reinforcement learning algorithms performing UAV navigation tasks in a 3D environment and has outstanding performance in algorithm effectiveness, with the average reward increasing by 9.35, the success rate of navigation tasks increasing by 14%, and the collision rate decreasing by 14%.

engineering, electrical & electronic,computer science, information systems,physics, applied

What problem does this paper attempt to address?

The paper attempts to address the problem of obstacle avoidance faced by Unmanned Aerial Vehicles (UAVs) during autonomous navigation in complex low-altitude environments. Existing reinforcement learning methods typically fix flight altitude and speed, which significantly reduces the difficulty of the algorithm. However, these methods perform poorly in complex low-altitude environments, especially in terms of obstacle avoidance capabilities. To improve the autonomous navigation and obstacle avoidance capabilities of UAVs in low-altitude environments, the authors propose an adaptive control method based on deep reinforcement learning.

Specifically, the main contributions of the paper include:
1. **Guided Attention Mechanism**: By using a guided attention mechanism, the UAV can flexibly switch decision focus between navigation tasks and obstacle avoidance tasks, thereby improving obstacle avoidance capabilities in complex environments.
2. **Speed Constraint Loss Function**: A new speed constraint loss function is introduced to help the UAV better learn speed control, further enhancing its obstacle avoidance capabilities.
3. **Improved TD3 Algorithm**: Combining TD3, LSTM, and attention mechanisms, a new algorithm framework (GARTD3) is proposed and validated in a complex 3D simulation environment, demonstrating the effectiveness of the method.

Through these innovations, the paper aims to address the inadequacies of existing methods in obstacle avoidance in complex low-altitude environments and improve the autonomous navigation performance of UAVs.