Abstract:Automatic collision avoidance decision making for vessels is a critical challenge in the development of autonomous ships and has become a central point of research in the maritime safety domain. Effective and systematic collision avoidance strategies significantly reduce the risk of vessel collisions, ensuring safe navigation. This study develops a multi-vessel automatic collision avoidance decision-making method based on deep reinforcement learning (DRL) and establishes a vessel behavior decision model. When designing the reward function for continuous action spaces, the criteria of the "Convention on the International Regulations for Preventing Collisions at Sea" (COLREGs) were adhered to, taking into account the vessel's collision risk under various encounter situations, real-world navigation practices, and navigational complexities. Furthermore, to enable the algorithm to precisely differentiate between collision avoidance and the navigation resumption phase in varied vessel encounter situations, this paper incorporated "collision avoidance decision making" and "course recovery decision making" as state parameters in the state set design, from which the respective objective functions were defined. To further enhance the algorithm's performance, techniques such as behavior cloning, residual networks, and CPU-GPU dual-core parallel processing modules were integrated. Through simulation experiments in the enhanced Imazu training environment, the practicality of the method, taking into account the effects of wind and ocean currents, was corroborated. The results demonstrate that the proposed algorithm can perform effective collision avoidance decision making in a range of vessel encounter situations, indicating its efficiency and robust generalization capabilities.

A path planning strategy unified with a COLREGS collision avoidance function based on deep reinforcement learning and artificial potential field

Path planning and dynamic collision avoidance algorithm under COLREGs via deep reinforcement learning

An Autonomous Path Planning Model for Unmanned Ships Based on Deep Reinforcement Learning

A Novel Reinforcement Learning Collision Avoidance Algorithm for USVs Based on Maneuvering Characteristics and COLREGs

COLREGs-Based Path Planning for USVs Using the Deep Reinforcement Learning Strategy

Real-time Planning and Collision Avoidance Control Method Based on Deep Reinforcement Learning

Deep Reinforcement Learning Based Path Planning and Collision Avoidance for Smart Ships in Complex Environments

A human-like collision avoidance method for USVs based on deep reinforcement learning and velocity obstacle

Research on collision avoidance algorithm of unmanned surface vehicle based on deep reinforcement learning

A COLREGs-Compliant Deep Reinforcement Learning Approach

Path Planning based on Deep Reinforcement Learning for Autonomous Underwater Vehicles under Ocean Current Disturbance

A path planning approach for unmanned surface vehicles based on dynamic and fast Q-learning

Collision Avoidance and Path Point Tracking Control for Underactuated Unmanned Surface Vehicles with Unknown Model Nonlinearity

Path planning of autonomous underwater vehicle in unknown environment based on improved deep reinforcement learning

COLREGs-abiding hybrid collision avoidance algorithm based on deep reinforcement learning for USVs

A Hybrid Path Planning Algorithm for Unmanned Surface Vehicles in Complex Environment with Dynamic Obstacles.

A new real‐time path planning for USV based on dynamic artificial potential field in complex environments

Collision avoidance decision-making strategy for multiple USVs based on Deep Reinforcement Learning algorithm

Proximal policy optimization with reciprocal velocity obstacle based collision avoidance path planning for multi-unmanned surface vehicles

A novel intelligent collision avoidance algorithm based on deep reinforcement learning approach for USV

A human-like collision avoidance method for autonomous ship with attention-based deep reinforcement learning