Abstract:With the growing utilization of UAV in reconnaissance, agriculture, logistics and entertainment, Autonomous collision avoidance during flight has become a necessary capability for modern UAV to detect the surrounding environment and guarantee their own safety. Autonomous obstacle avoidance is a typical agent decision-making problem. Unfortunately, existing traditional decision-making methods perform poorly in this specific realm, In particular, it is unable to meet the requirements of three-dimensional obstacle avoidance of UAV, so we introduce the deep reinforcement learning (DRL) technique into autonomous obstacle avoidance. We model the obstacle avoidance process as a Markov Decision Process and introduce a structure composed of double joint neural network estimators as the decision-maker, whose input is omnidirectional sonar readings and whose output is a value function estimating future rewards. Also, we propose an adaption in the procedure of memory replay to optimize the sampling, where we assign weights to the transitions and sample them accordingly. Our method is applied in a 3-dimensional physic environment, which contains both random dynamic obstacles and floating bouncing obstacles. The goal of the drone is to reach the terminal point without crash. Double Q Learning method with priority sampling, by comparison, achieves the most excellent performance in our simulation. Compared with the traditional algorithms, the proposed algorithm not only ensures the quality of decision making, enabling the agent to learn the optimal strategy, but also effectively improves the performance of the task and the efficiency of decision making. Simulation results demonstrate its effectiveness.

Design and Field Test of Collision Avoidance Method with Prediction for USVs: A Deep Deterministic Policy Gradient Approach

A Balanced Collision Avoidance Algorithm for USVs in Complex Environment: A Deep Reinforcement Learning Approach

Research on collision avoidance algorithm of unmanned surface vehicle based on deep reinforcement learning

Obstacle avoidance USV in multi-static obstacle environments based on a deep reinforcement learning approach

A human-like collision avoidance method for USVs based on deep reinforcement learning and velocity obstacle

AUV Collision Avoidance Planning Method Based on Deep Deterministic Policy Gradient

Collision avoidance decision-making strategy for multiple USVs based on Deep Reinforcement Learning algorithm

A novel intelligent collision avoidance algorithm based on deep reinforcement learning approach for USV

Collision-avoidance under COLREGS for Unmanned Surface Vehicles Via Deep Reinforcement Learning

COLREGs-abiding hybrid collision avoidance algorithm based on deep reinforcement learning for USVs

DRL-based target interception strategy design for an underactuated USV without obstacle collision

Proximal policy optimization with reciprocal velocity obstacle based collision avoidance path planning for multi-unmanned surface vehicles

Research on Unmanned Surface Vehicle Collision Avoidance Based on Deep Reinforcement Learning

Real-time Planning and Collision Avoidance Control Method Based on Deep Reinforcement Learning

Path planning and dynamic collision avoidance algorithm under COLREGs via deep reinforcement learning

Deep-Reinforcement-Learning-Based Collision Avoidance of Autonomous Driving System for Vulnerable Road User Safety

Intelligent Decision-Making for 3-Dimensional Dynamic Obstacle Avoidance of UAV Based on Deep Reinforcement Learning

Reinforcement Learning Based Obstacle Avoidance for AUV Swarm in Dynamic Ocean Environment

Dynamic Obstacle Avoidance for USVs Using Cross-Domain Deep Reinforcement Learning and Neural Network Model Predictive Controller

Confrontation and Obstacle-Avoidance of Unmanned Vehicles Based on Progressive Reinforcement Learning.

A Reinforcement Learning-based Decentralized Method of Avoiding Multi-UAV Collision in 3-D Airspace