Abstract:Autonomous collision avoidance technology provides an intelligent method for unmanned surface vehicles’ (USVs) safe and efficient navigation. In this paper, the USV collision avoidance problem under the constraint of the international regulations for preventing collisions at sea (COLREGs) was studied. Here, a reinforcement learning collision avoidance (RLCA) algorithm is proposed that complies with USV maneuverability. Notably, the reinforcement learning agent does not require any prior knowledge about USV collision avoidance from humans to learn collision avoidance motions well. The double-DQN method was used to reduce the overestimation of the action-value function. A dueling network architecture was adopted to clearly distinguish the difference between a great state and an excellent action. Aiming at the problem of agent exploration, a method based on the characteristics of USV collision avoidance, the category-based exploration method, can improve the exploration ability of the USV. Because a large number of turning behaviors in the early steps may affect the training, a method to discard some of the transitions was designed, which can improve the effectiveness of the algorithm. A finite Markov decision process (MDP) that conforms to the USVs’ maneuverability and COLREGs was used for the agent training. The RLCA algorithm was tested in a marine simulation environment in many different USV encounters, which showed a higher average reward. The RLCA algorithm bridged the divide between USV navigation status information and collision avoidance behavior, resulting in successfully planning a safe and economical path to the terminal.

A Greedy Navigation and Subtle Obstacle Avoidance Algorithm for USV Using Reinforcement Learning

Obstacle Avoidance Strategy with Path-following for USV Based on Obstacle-guidance Model Using 2D-Lidar in Urban River Conditions

Obstacle avoidance USV in multi-static obstacle environments based on a deep reinforcement learning approach

An adaptive obstacle avoidance algorithm for unmanned surface vehicle in complicated marine environments

Collision Avoidance and Path Point Tracking Control for Underactuated Unmanned Surface Vehicles with Unknown Model Nonlinearity

A Stable Local Reactive Obstacle Avoidance Algorithm for High-Speed Unmanned Surface Vehicle

A novel intelligent collision avoidance algorithm based on deep reinforcement learning approach for USV

A Real-time Algorithm for USV Navigation Based on Deep Reinforcement Learning

Path Planning of Unmanned Surface Vehicle Based on Improved Q-Learning Algorithm

A Balanced Collision Avoidance Algorithm for USVs in Complex Environment: A Deep Reinforcement Learning Approach

Proximal policy optimization with reciprocal velocity obstacle based collision avoidance path planning for multi-unmanned surface vehicles

Pursuit-Evasion Game of Unmanded Surface Vehicles Based on Deep Reinforcement Learning

Autonomous Obstacle Avoidance Algorithm for Unmanned Surface Vehicles Based on an Improved Velocity Obstacle Method

A Novel Reinforcement Learning Collision Avoidance Algorithm for USVs Based on Maneuvering Characteristics and COLREGs

Obstacle avoidance for environmentally-driven USVs based on deep reinforcement learning in large-scale uncertain environments

A human-like collision avoidance method for USVs based on deep reinforcement learning and velocity obstacle

Real-time Planning and Collision Avoidance Control Method Based on Deep Reinforcement Learning

Intelligent Collision Avoidance Algorithms for USVs Via Deep Reinforcement Learning under COLREGs

Multi-USV Dynamic Navigation and Target Capture: A Guided Multi-Agent Reinforcement Learning Approach

Greedy Mechanism Based Particle Swarm Optimization for Path Planning Problem of an Unmanned Surface Vehicle

Adaptive Optimal Surrounding Control of Multiple Unmanned Surface Vessels via Actor-Critic Reinforcement Learning