Memory-based Deep Reinforcement Learning Method for Obstacle Avoidance in UAV

Sindhu Padakandla,Abhik Singla,S. Bhatnagar

Abstract:Abhik Singla Indian Institute of Science abhiksingla10@gmail.com Sindhu Padakandla Indian Institute of Science sindhupr@iisc.ac.in Shalabh Bhatnagar Indian Institute of Science shalabh@iisc.ac.in

Computer Science,Engineering

What problem does this paper attempt to address?

Memory-Based Deep Reinforcement Learning for Obstacle Avoidance in UAV With Limited Environment Knowledge

Abhik Singla,Sindhu Padakandla,Shalabh Bhatnagar

DOI: https://doi.org/10.1109/tits.2019.2954952

IF: 8.5

2021-01-01

IEEE Transactions on Intelligent Transportation Systems

Abstract:This paper presents our method for enabling a UAV quadrotor, equipped with a monocular camera, to autonomously avoid collisions with obstacles in unstructured and unknown indoor environments. When compared to obstacle avoidance in ground vehicular robots, UAV navigation brings in additional challenges because the UAV motion is no more constrained to a well-defined indoor ground or street environment. Unlike ground vehicular robots, a UAV has to navigate across more types of obstacles - for e.g., objects like decorative items, furnishings, ceiling fans, sign-boards, tree branches, etc., are also potential obstacles for a UAV. Thus, methods of obstacle avoidance developed for ground robots are clearly inadequate for UAV navigation. Current control methods using monocular images for UAV obstacle avoidance are heavily dependent on environment information. These controllers do not fully retain and utilize the extensively available information about the ambient environment for decision making. We propose a deep reinforcement learning based method for UAV obstacle avoidance (OA) which is capable of doing exactly the same. The crucial idea in our method is the concept of partial observability and how UAVs can retain relevant information about the environment structure to make better future navigation decisions. Our OA technique uses recurrent neural networks with temporal attention and provides better results compared to prior works in terms of distance covered without collisions. In addition, our technique has a high inference rate and reduces power wastage as it minimizes oscillatory motion of UAV.

engineering, electrical & electronic,transportation science & technology, civil
A Vision Based Deep Reinforcement Learning Algorithm for UAV Obstacle Avoidance

Jeremy Roghair,Kyungtae Ko,Amir Ehsan Niaraki Asli,Ali Jannesari

DOI: https://doi.org/10.48550/arXiv.2103.06403

2021-03-11

Abstract:Integration of reinforcement learning with unmanned aerial vehicles (UAVs) to achieve autonomous flight has been an active research area in recent years. An important part focuses on obstacle detection and avoidance for UAVs navigating through an environment. Exploration in an unseen environment can be tackled with Deep Q-Network (DQN). However, value exploration with uniform sampling of actions may lead to redundant states, where often the environments inherently bear sparse rewards. To resolve this, we present two techniques for improving exploration for UAV obstacle avoidance. The first is a convergence-based approach that uses convergence error to iterate through unexplored actions and temporal threshold to balance exploration and exploitation. The second is a guidance-based approach using a Domain Network which uses a Gaussian mixture distribution to compare previously seen states to a predicted next state in order to select the next action. Performance and evaluation of these approaches were implemented in multiple 3-D simulation environments, with variation in complexity. The proposed approach demonstrates a two-fold improvement in average rewards compared to state of the art.

Artificial Intelligence,Computer Vision and Pattern Recognition,Robotics
Deep Reinforcement Learning for Vision-Based Navigation of UAVs in Avoiding Stationary and Mobile Obstacles

Amudhini P. Kalidas,Christy Jackson Joshua,Shakila Basheer,Senthilkumar Mohan,Sapiah Sakri,Abdul Quadir Md

DOI: https://doi.org/10.3390/drones7040245

IF: 5.532

2023-04-01

Drones

Abstract:Unmanned Aerial Vehicles (UAVs), also known as drones, have advanced greatly in recent years. There are many ways in which drones can be used, including transportation, photography, climate monitoring, and disaster relief. The reason for this is their high level of efficiency and safety in all operations. While the design of drones strives for perfection, it is not yet flawless. When it comes to detecting and preventing collisions, drones still face many challenges. In this context, this paper describes a methodology for developing a drone system that operates autonomously without the need for human intervention. This study applies reinforcement learning algorithms to train a drone to avoid obstacles autonomously in discrete and continuous action spaces based solely on image data. The novelty of this study lies in its comprehensive assessment of the advantages, limitations, and future research directions of obstacle detection and avoidance for drones, using different reinforcement learning techniques. This study compares three different reinforcement learning strategies—namely, Deep Q-Networks (DQN), Proximal Policy Optimization (PPO), and Soft Actor-Critic (SAC)—that can assist in avoiding obstacles, both stationary and moving; however, these strategies have been more successful in drones. The experiment has been carried out in a virtual environment made available by AirSim. Using Unreal Engine 4, the various training and testing scenarios were created for understanding and analyzing the behavior of RL algorithms for drones. According to the training results, SAC outperformed the other two algorithms. PPO was the least successful among the algorithms, indicating that on-policy algorithms are ineffective in extensive 3D environments with dynamic actors. DQN and SAC, two off-policy algorithms, produced encouraging outcomes. However, due to its constrained discrete action space, DQN may not be as advantageous as SAC in narrow pathways and twists. Concerning further findings, when it comes to autonomous drones, off-policy algorithms, such as DQN and SAC, perform more effectively than on-policy algorithms, such as PPO. The findings could have practical implications for the development of safer and more efficient drones in the future.
Deep-Reinforcement-Learning-Based Collision Avoidance in UAV Environment

Sihem Ouahouah,Miloud Bagaa,Jonathan Prados-Garzon,Tarik Taleb

DOI: https://doi.org/10.1109/jiot.2021.3118949

IF: 10.6

2022-03-15

IEEE Internet of Things Journal

Abstract:Unmanned aerial vehicles (UAVs) have recently attracted both academia and industry representatives due to their utilization in tremendous emerging applications. Most UAV applications adopt visual line of sight (VLOS) due to ongoing regulations. There is a consensus between industry for extending UAVs' commercial operations to cover the urban and populated area-controlled airspace beyond VLOS (BVLOS). There is ongoing regulation for enabling BVLOS UAV management. Regrettably, this comes with unavoidable challenges related to UAVs' autonomy for detecting and avoiding static and mobile objects. An intelligent component should either be deployed onboard the UAV or at a multiaccess-edge computing (MEC) that can read the gathered data from different UAV's sensors, process them, and then make the right decision to detect and avoid the physical collision. The sensing data should be collected using various sensors but not limited to Lidar, depth camera, video, or ultrasonic. This article proposes probabilistic and deep-reinforcement-learning (DRL)-based algorithms for avoiding collisions while saving energy consumption. The proposed algorithms can be either run on top of the UAV or at the MEC according to the UAV capacity and the task overhead. We have designed and developed our algorithms to work for any environment without a need for any prior knowledge. The proposed solutions have been evaluated in a harsh environment that consists of many UAVs moving randomly in a small area without any correlation. The obtained results demonstrated the efficiency of these solutions for avoiding the collision while saving energy consumption in familiar and unfamiliar environments.

computer science, information systems,telecommunications,engineering, electrical & electronic
Autonomous obstacle avoidance of UAV based on deep reinforcement learning

Han Li,Z. Meng,Guizhen Yu,Zhangyu Wang,Songyue Yang

DOI: https://doi.org/10.3233/jifs-211192

2021-08-28

Journal of Intelligent & Fuzzy Systems

Abstract:In the intelligent unmanned systems, unmanned aerial vehicle (UAV) obstacle avoidance technology is the core and primary condition. Traditional algorithms are not suitable for obstacle avoidance in complex and changeable environments based on the limited sensors on UAVs. In this article, we use an end-to-end deep reinforcement learning (DRL) algorithm to achieve the UAV autonomously avoid obstacles. For the problem of slow convergence in DRL, a Multi-Branch (MB) network structure is proposed to ensure that the algorithm can get good performance in the early stage; for non-optimal decision-making problems caused by overestimation, the Revise Q-value (RQ) algorithm is proposed to ensure that the agent can choose the optimal strategy for obstacle avoidance. According to the flying characteristics of the rotor UAV, we build a V-Rep 3D physical simulation environment to test the obstacle avoidance performance. And experiments show that the improved algorithm can accelerate the convergence speed of agent and the average return of the round is increased by 25%.

Computer Science,Engineering
Vision Based Drone Obstacle Avoidance by Deep Reinforcement Learning

Zhihan Xue,Tad Gonsalves

DOI: https://doi.org/10.3390/ai2030023

2021-08-19

AI

Abstract:Research on autonomous obstacle avoidance of drones has recently received widespread attention from researchers. Among them, an increasing number of researchers are using machine learning to train drones. These studies typically adopt supervised learning or reinforcement learning to train the networks. Supervised learning has a disadvantage in that it takes a significant amount of time to build the datasets, because it is difficult to cover the complex and changeable drone flight environment in a single dataset. Reinforcement learning can overcome this problem by using drones to learn data in the environment. However, the current research results based on reinforcement learning are mainly focused on discrete action spaces. In this way, the movement of drones lacks precision and has somewhat unnatural flying behavior. This study aims to use the soft-actor-critic algorithm to train a drone to perform autonomous obstacle avoidance in continuous action space using only the image data. The algorithm is trained and tested in a simulation environment built by Airsim. The results show that our algorithm enables the UAV to avoid obstacles in the training environment only by inputting the depth map. Moreover, it also has a higher obstacle avoidance rate in the reconfigured environment without retraining.
UAV Obstacle Avoidance by Human-in-the-Loop Reinforcement in Arbitrary 3D Environment

Xuyang Li,Jianwu Fang,Kai Du,Kuizhi Mei,Jianru Xue

DOI: https://doi.org/10.48550/arXiv.2304.05959

2023-04-07

Abstract:This paper focuses on the continuous control of the unmanned aerial vehicle (UAV) based on a deep reinforcement learning method for a large-scale 3D complex environment. The purpose is to make the UAV reach any target point from a certain starting point, and the flying height and speed are variable during navigation. In this work, we propose a deep reinforcement learning (DRL)-based method combined with human-in-the-loop, which allows the UAV to avoid obstacles automatically during flying. We design multiple reward functions based on the relevant domain knowledge to guide UAV navigation. The role of human-in-the-loop is to dynamically change the reward function of the UAV in different situations to suit the obstacle avoidance of the UAV better. We verify the success rate and average step size on urban, rural, and forest scenarios, and the experimental results show that the proposed method can reduce the training convergence time and improve the efficiency and accuracy of navigation tasks. The code is available on the website <a class="link-external link-https" href="https://github.com/Monnalo/UAV_navigation" rel="external noopener nofollow">this https URL</a>.

Robotics,Artificial Intelligence
Multi-UAV Autonomous Obstacle Avoidance Based on Reinforcement Learning

Zheng Li,Yanhui Wang,Jinna Li

DOI: https://doi.org/10.23919/CCC58697.2023.10240772

2023-07-24

Abstract:Obstacle avoidance is a necessary behaviour to ensure the safety of UAV. In this paper, aiming at the problem of autonomous learning and obstacle avoidance of multiple UAVs in the multi-obstacle map environment, an obstacle avoidance method of UAVs based on an improved reward deep Q learning network is proposed. According to the dynamics model of UAV, the three-dimensional dynamic equation is established, and the combination of pitch angle and heading angle constructs the action space of UAV. a new reward evaluation method, which adaptively adjusts the weight of the reward according to the distance between the UAV and the obstacle, is developed thus improving the performance of the UAV. Thus, the obstacle avoidance performance of multiple UAVs in unknown environment can be effectively improved. Finally, by comparing the traditional deep Q learning network, the simulation results show that the algorithm in this paper is reasonable, and UAVs can successfully achieve obstacle avoidance in unknown environment.

Computer Science,Engineering
Integrated moment-based LGMD and deep reinforcement learning for UAV obstacle avoidance

Lei He,N. Aouf,J. Whidborne,Bifeng Song

DOI: https://doi.org/10.1109/ICRA40945.2020.9197152

2020-05-01

Abstract:In this paper, a bio-inspired monocular vision perception method combined with a learning-based reaction local planner for obstacle avoidance of micro UAVs is presented. The system is more computationally efficient than other vision-based perception and navigation methods such as SLAM and optical flow because it does not need to calculate accurate distances. To improve the robustness of perception against illuminance change, the input image is remapped using image moment which is independent of illuminance variation. After perception, a local planner is trained using deep reinforcement learning for mapless navigation. The proposed perception and navigation methods are evaluated in some realistic simulation environments. The result shows that this light-weight monocular perception and navigation system works well in different complex environments without accurate depth information.

Computer Science,Engineering,Environmental Science
Interpretable UAV Collision Avoidance using Deep Reinforcement Learning

Deepak-George Thomas,Daniil Olshanskyi,Karter Krueger,Tichakorn Wongpiromsarn,Ali Jannesari

DOI: https://doi.org/10.48550/arXiv.2105.12254

2021-06-05

Abstract:The significant components of any successful autonomous flight system are task completion and collision avoidance. Most deep learning algorithms successfully execute these aspects under the environment and conditions they are trained. However, they fail when subjected to novel environments. This paper presents an autonomous multi-rotor flight algorithm, using Deep Reinforcement Learning augmented with Self-Attention Models, that can effectively reason when subjected to varying inputs. In addition to their reasoning ability, they are also interpretable, enabling it to be used under real-world conditions. We have tested our algorithm under different weather conditions and environments and found it robust compared to conventional Deep Reinforcement Learning algorithms.

Robotics,Machine Learning
Application of Deep Reinforcement Learning in UAVs: A Review

Ruihui Wang,Li Xu

DOI: https://doi.org/10.1109/ccdc55256.2022.10034357

2022-01-01

Abstract:Deep reinforcement learning is one of the most important branches in the field of artificial intelligence. It has strong high-latitude data processing capabilities and is mainly used to solve control and decision-making problems. Many problems in the field of unmanned aerial vehicle(UAV) involve control and decision-making, so deep reinforcement learning has a wide range of applications in the field of UAVs. First, this article introduces the principles of classic reinforcement learning and extends to the principles of deep reinforcement learning. Secondly, this article describes the control and decision-making problems in the field of UAVs, and further introduces the application of deep reinforcement learning to these problems. Finally, this article looks forward to the future development direction of deep reinforcement learning in this field, and provides references for future related research.
Autonomous UAV Navigation with Adaptive Control Based on Deep Reinforcement Learning

Yongfeng Yin,Zhetao Wang,Lili Zheng,Qingran Su,Yang Guo

DOI: https://doi.org/10.3390/electronics13132432

IF: 2.9

2024-06-22

Electronics

Abstract:Unmanned aerial vehicle (UAV) navigation plays a crucial role in its ability to perform autonomous missions in complex environments. Most of the existing reinforcement learning methods to solve the UAV navigation problem fix the flight altitude and velocity, which largely reduces the difficulty of the algorithm. But the methods without adaptive control are not suitable in low-altitude environments with complex situations, generally suffering from weak obstacle avoidance. Some UAV navigation studies with adaptive flight only have weak obstacle avoidance capabilities. To address the problem of UAV navigation in low-altitude environments, we construct autonomous UAV navigation in 3D environments with adaptive control as a Markov decision process and propose a deep reinforcement learning algorithm. To solve the problem of weak obstacle avoidance, we creatively propose the guide attention method to make a UAV's decision focus shift between the navigation task and obstacle avoidance task according to changes in the obstacle. We raise a novel velocity-constrained loss function and add it to the original actor loss to improve the UAV's velocity control capability. Simulation experiment results demonstrate that our algorithm outperforms some of the state-of-the-art deep reinforcement learning algorithms performing UAV navigation tasks in a 3D environment and has outstanding performance in algorithm effectiveness, with the average reward increasing by 9.35, the success rate of navigation tasks increasing by 14%, and the collision rate decreasing by 14%.

engineering, electrical & electronic,computer science, information systems,physics, applied
A Learning Method for AUV Collision Avoidance Through Deep Reinforcement Learning

Jian Xu,Fei Huang,Di Wu,Yunfei Cui,Zheping Yan,Xue Du

DOI: https://doi.org/10.1016/j.oceaneng.2022.112038

IF: 5

2022-01-01

Ocean Engineering

Abstract:This paper investigates the collision avoidance problem for autonomous underwater vehicle (AUV) with limited detection range via deep reinforcement learning method in an unknown underwater environment. Firstly, to deal with both unknown static and dynamic obstacles, we introduce two different event-triggered mechanism to generate obstacle-related states and reward the safe collision avoidance action of AUV, respectively. Secondly, the complete state space is customized by combining the state of AUV itself and the state related to a target area. In addition, considering the task requirement of reaching a target area with the shortest path, the complete reward function is designed. Then, we propose a novel event-triggered soft actor–critic (ET-SAC) algorithm for AUV collision avoidance, and give its detailed pseudo code and implementation architecture. Finally, training and evaluation are carried out in the simulation platform we designed, including unknown static obstacle environment and unknown dynamic obstacle environment in 2D/3D space, and the results confirm that the proposed algorithm of this paper is feasible and effective for AUV to avoid collision safely.
Robotic vision based obstacle avoidance for navigation of unmanned aerial vehicle using fuzzy rule based optimal deep learning model

K. N. V. Suresh Varma,S. Lalitha Kumari

DOI: https://doi.org/10.1007/s12065-023-00881-9

2023-09-28

Evolutionary Intelligence

Abstract:Recently, the topics associated with robotics are becoming an interesting research area. Meanwhile, intelligent mobile robots or unmanned aerial vehicle (UAV) has great acceptance; however, the navigation and control of the devices are more complex. The shortage of dealing with static obstacles and evading them due to secure and safe routing is a fundamental need of the system. Therefore, an effectual robotic obstacle avoidance mechanism using deep learning (DL) model optimized with a swarm intelligence procedure is proposed. The main aim of this method is to avoid collision with static obstacles and attain the target in an optimal path. At first, the proposed method initializes the obstacles and target places for mobile robot navigation, and their positions are collected. An innovative path planning approach is presented using adaptive beetle swarm levy flight optimization (BSLFO) for optimal path calculation. Here, distance and path smoothness parameters are considered for optimal planning by the proposed BSLFO approach. The mobile robots are moved to the best positions obtained by the BSLFO approach. Additionally, optimal robots’ movement with collision avoidance has been performed by employing a fuzzy rule based optimal deep neural network (FRDNN). The proposed method is implemented in the MATLAB platform and evaluates the performance in terms of performance metrics, including accuracy, recall, precision, and planning time. The maximum accuracy reached by the proposed method for static and dynamic obstacles is 98.62, which offers better obstacle avoidance and effective path planning. Consequently, the overall experimental outcomes proved that the proposed model outperforms the existing models.
Obstacle Avoidance for UAS in Continuous Action Space Using Deep Reinforcement Learning

Jueming Hu,Xuxi Yang,Weichang Wang,Peng Wei,Lei Ying,Yongming Liu

DOI: https://doi.org/10.1109/access.2022.3201962

IF: 3.9

2022-09-07

IEEE Access

Abstract:Obstacle avoidance for small unmanned aircraft is vital for the safety of future urban air mobility (UAM) and Unmanned Aircraft System (UAS) Traffic Management (UTM). There are a variety of techniques for real-time robust drone guidance, but numerous of them solve in discretized airspace and control, which would require an additional path smoothing step to provide flexible commands for UAS. To deliver safe and computationally efficient guidance for UAS operations, we explore the use of a deep reinforcement learning algorithm based on Proximal Policy Optimization (PPO) to lead autonomous UAS to their destinations while bypassing obstacles through continuous control. The proposed scenario state representation and reward function can map the continuous state space to continuous control for both heading angle and speed. To verify the effectiveness of the proposed learning framework, we conducted numerical experiments with static and moving obstacles. Uncertainties associated with the environments and safety operation bounds are investigated in detail. Results show that the proposed model is able to provide accurate and robust guidance and resolve conflict with a success rate of over 99%.

computer science, information systems,telecommunications,engineering, electrical & electronic
The Algorithm for UAV Obstacle Avoidance and Route Planning Based on Reinforcement Learning

Jiantong Liu,Zhengjie Wang,Zhide Zhang

DOI: https://doi.org/10.1007/978-981-15-0474-7_70

2019-12-04

Abstract:An algorithm for UAV collision avoidance based on the reinforcement learning is used for small fixed-wing unmanned aerial vehicles (UAVs). The proposed algorithm realized the obstacle avoidance of UAV in unknown environment and the result is close to the global optimal path. The simulation results show that the collision avoidance algorithm can adapt to various complex environments. Meanwhile, the UAV can quickly get close to the target while avoiding obstacles.
Advancements in UAV Path Planning: A Deep Reinforcement Learning Approach with Soft Actor-Critic for Enhanced Navigation

Jingrui Guo,Guanzhong Zhou,Hailong Huang,Chao Huang

DOI: https://doi.org/10.1142/s2301385025500669

2024-09-18

Unmanned Systems

Abstract:Unmanned Systems, Ahead of Print. This paper tackles the intricate challenge of autonomous navigation for Unmanned Aerial Vehicles (UAVs) through dynamically changing environments. We focus on a sophisticated Deep Reinforcement Learning (DRL) approach using the Soft Actor-Critic (SAC) algorithm, optimized for UAV path planning within a continuous action space. This methodology leverages environmental image data to enhance the precision of flight maneuvers and effective obstacle avoidance. Our approach, validated through extensive simulations in Gazebo and field tests, demonstrates the algorithm's efficacy in enabling UAVs to adeptly navigate obstacles using depth maps. The study further explores the robustness of the SAC algorithm by comparing it with traditional DRL methods, emphasizing its superior performance in real-world applications. This research contributes significantly to advancing UAV technology, particularly in autonomous motion planning, by integrating cutting-edge machine learning techniques. The video link is: https://www.youtube.com/watch?v=Nd_aMzejNXY.
UAV Multi-Dynamic Target Interception: A Hybrid Intelligent Method Using Deep Reinforcement Learning and Fuzzy Logic

Bingze Xia,Iraj Mantegh,Wenfang Xie

DOI: https://doi.org/10.3390/drones8060226

IF: 5.532

2024-05-29

Drones

Abstract:With the rapid development of Artificial Intelligence, AI-enabled Uncrewed Aerial Vehicles have garnered extensive attention since they offer an accessible and cost-effective solution for executing tasks in unknown or complex environments. However, developing secure and effective AI-based algorithms that empower agents to learn, adapt, and make precise decisions in dynamic situations continues to be an intriguing area of study. This paper proposes a hybrid intelligent control framework that integrates an enhanced Soft Actor–Critic method with a fuzzy inference system, incorporating pre-defined expert experience to streamline the learning process. Additionally, several practical algorithms and approaches within this control system are developed. With the synergy of these innovations, the proposed method achieves effective real-time path planning in unpredictable environments under a model-free setting. Crucially, it addresses two significant challenges in RL: dynamic-environment problems and multi-target problems. Diverse scenarios incorporating actual UAV dynamics were designed and simulated to validate the performance in tracking multiple mobile intruder aircraft. A comprehensive analysis and comparison of methods relying solely on RL and other influencing factors, as well as a controller feasibility assessment for real-world flight tests, are conducted, highlighting the advantages of the proposed hybrid architecture. Overall, this research advances the development of AI-driven approaches for UAV safe autonomous navigation under demanding airspace conditions and provides a viable learning-based control solution for different types of robots.
ENHANCING AUTONOMOUS NAVIGATION AND COLLISION AVOIDANCE IN DRONE TECHNOLOGY USING DEEP REINFORCEMENT LEARNING

Vikash Kumar,Lokesh Kumar,Anuj Kumar,Naman Chauhan,Bhanu Pratap Singh

DOI: https://doi.org/10.30780/ijtrs.v09.i04.003

2024-04-25

International Journal of Technical Research & Science

Abstract:Drone technology is advancing quickly; effective and secure autonomous navigation systems are crucial. This paper presents a novel approach to enhance autonomous navigation and collision avoidance in drone technology using Deep Reinforcement [11] Learning (DRL). Our objective is to apply DRL algorithms to improve drone decision-making abilities, enabling them to navigate complex environments more effectively and safely.
Deep-reinforcement-learning-based UAV autonomous navigation and collision avoidance in unknown environments

WANG Fei,ZHU Xiaoping,ZHOU Zhou,TANG Yang,Fei WANG,Xiaoping ZHU,Zhou ZHOU,Yang TANG

DOI: https://doi.org/10.1016/j.cja.2023.09.033

IF: 5.7

2023-10-18

Chinese Journal of Aeronautics

Abstract:In some military application scenarios, Unmanned Aerial Vehicles (UAVs) need to perform missions with the assistance of on-board cameras when radar is not available and communication is interrupted, which brings challenges for UAV autonomous navigation and collision avoidance. In this paper, an improved deep-reinforcement-learning algorithm, Deep Q-Network with a Faster R-CNN model and a Data Deposit Mechanism (FRDDM-DQN), is proposed. A Faster R-CNN model (FR) is introduced and optimized to obtain the ability to extract obstacle information from images, and a new replay memory Data Deposit Mechanism (DDM) is designed to train an agent with a better performance. During training, a two-part training approach is used to reduce the time spent on training as well as retraining when the scenario changes. In order to verify the performance of the proposed method, a series of experiments, including training experiments, test experiments, and typical episodes experiments, is conducted in a 3D simulation environment. Experimental results show that the agent trained by the proposed FRDDM-DQN has the ability to navigate autonomously and avoid collisions, and performs better compared to the FR-DQN, FR-DDQN, FR-Dueling DQN, YOLO-based YDDM-DQN, and original FR output-based FR-ODQN.

engineering, aerospace

Memory-based Deep Reinforcement Learning Method for Obstacle Avoidance in UAV

Memory-Based Deep Reinforcement Learning for Obstacle Avoidance in UAV With Limited Environment Knowledge

A Vision Based Deep Reinforcement Learning Algorithm for UAV Obstacle Avoidance

Deep Reinforcement Learning for Vision-Based Navigation of UAVs in Avoiding Stationary and Mobile Obstacles

Deep-Reinforcement-Learning-Based Collision Avoidance in UAV Environment

Autonomous obstacle avoidance of UAV based on deep reinforcement learning

Vision Based Drone Obstacle Avoidance by Deep Reinforcement Learning

UAV Obstacle Avoidance by Human-in-the-Loop Reinforcement in Arbitrary 3D Environment

Multi-UAV Autonomous Obstacle Avoidance Based on Reinforcement Learning

Integrated moment-based LGMD and deep reinforcement learning for UAV obstacle avoidance

Interpretable UAV Collision Avoidance using Deep Reinforcement Learning

Application of Deep Reinforcement Learning in UAVs: A Review

Autonomous UAV Navigation with Adaptive Control Based on Deep Reinforcement Learning

A Learning Method for AUV Collision Avoidance Through Deep Reinforcement Learning

Robotic vision based obstacle avoidance for navigation of unmanned aerial vehicle using fuzzy rule based optimal deep learning model

Obstacle Avoidance for UAS in Continuous Action Space Using Deep Reinforcement Learning

The Algorithm for UAV Obstacle Avoidance and Route Planning Based on Reinforcement Learning

Advancements in UAV Path Planning: A Deep Reinforcement Learning Approach with Soft Actor-Critic for Enhanced Navigation

UAV Multi-Dynamic Target Interception: A Hybrid Intelligent Method Using Deep Reinforcement Learning and Fuzzy Logic

ENHANCING AUTONOMOUS NAVIGATION AND COLLISION AVOIDANCE IN DRONE TECHNOLOGY USING DEEP REINFORCEMENT LEARNING

Deep-reinforcement-learning-based UAV autonomous navigation and collision avoidance in unknown environments