Abstract:Automatic collision avoidance decision making for vessels is a critical challenge in the development of autonomous ships and has become a central point of research in the maritime safety domain. Effective and systematic collision avoidance strategies significantly reduce the risk of vessel collisions, ensuring safe navigation. This study develops a multi-vessel automatic collision avoidance decision-making method based on deep reinforcement learning (DRL) and establishes a vessel behavior decision model. When designing the reward function for continuous action spaces, the criteria of the “Convention on the International Regulations for Preventing Collisions at Sea” (COLREGs) were adhered to, taking into account the vessel’s collision risk under various encounter situations, real-world navigation practices, and navigational complexities. Furthermore, to enable the algorithm to precisely differentiate between collision avoidance and the navigation resumption phase in varied vessel encounter situations, this paper incorporated “collision avoidance decision making” and “course recovery decision making” as state parameters in the state set design, from which the respective objective functions were defined. To further enhance the algorithm’s performance, techniques such as behavior cloning, residual networks, and CPU-GPU dual-core parallel processing modules were integrated. Through simulation experiments in the enhanced Imazu training environment, the practicality of the method, taking into account the effects of wind and ocean currents, was corroborated. The results demonstrate that the proposed algorithm can perform effective collision avoidance decision making in a range of vessel encounter situations, indicating its efficiency and robust generalization capabilities.

Reinforcement Learning from Demonstrations by Novel Interactive Expert and Application to Automatic Berthing Control Systems for Unmanned Surface Vessel

Self-Guided Actor-Critic: Reinforcement Learning from Adaptive Expert Demonstrations

Exploration-efficient Deep Reinforcement Learning with Demonstration Guidance for Robot Control

Motion Planning Using Reinforcement Learning Method for Underactuated Ship Berthing.

A Deep Reinforcement Learning Algorithm with Expert Demonstrations and Supervised Loss and Its Application in Autonomous Driving

Distributional Reinforcement Learning based Integrated Decision Making and Control for Autonomous Surface Vehicles

Intelligent AUV Surfacing Control in Network Attack Scenario

Aquatic Navigation: A Challenging Benchmark for Deep Reinforcement Learning

Harnessing traditional controllers for fast-track training of deep reinforcement learning control strategies

Optimizing Multi-Vessel Collision Avoidance Decision Making for Autonomous Surface Vessels: A COLREGs-Compliant Deep Reinforcement Learning Approach

Deep Reinforcement Learning Based Tracking Control of an Autonomous Surface Vessel in Natural Waters

A COLREGs-Compliant Deep Reinforcement Learning Approach

Decision-Making for the Autonomous Navigation of Maritime Autonomous Surface Ships Based on Scene Division and Deep Reinforcement Learning

AI on the Water: Applying DRL to Autonomous Vessel Navigation

Active Deep Q-learning with Demonstration

Continuous Control for Autonomous Underwater Vehicle Path Following Using Deep Interactive Reinforcement Learning

Tracking Control of AUV Via Novel Soft Actor-Critic and Suboptimal Demonstrations

A Novel Deep Reinforcement Learning for POMDP-based Autonomous Ship Collision Decision-Making

Deep Reinforcement Learning for Continuous Docking Control of Autonomous Underwater Vehicles: A Benchmarking Study

Deep Interactive Reinforcement Learning for Path Following of Autonomous Underwater Vehicle

Evaluating Robustness of Reinforcement Learning Algorithms for Autonomous Shipping