Abstract:Automatic collision avoidance decision making for vessels is a critical challenge in the development of autonomous ships and has become a central point of research in the maritime safety domain. Effective and systematic collision avoidance strategies significantly reduce the risk of vessel collisions, ensuring safe navigation. This study develops a multi-vessel automatic collision avoidance decision-making method based on deep reinforcement learning (DRL) and establishes a vessel behavior decision model. When designing the reward function for continuous action spaces, the criteria of the "Convention on the International Regulations for Preventing Collisions at Sea" (COLREGs) were adhered to, taking into account the vessel's collision risk under various encounter situations, real-world navigation practices, and navigational complexities. Furthermore, to enable the algorithm to precisely differentiate between collision avoidance and the navigation resumption phase in varied vessel encounter situations, this paper incorporated "collision avoidance decision making" and "course recovery decision making" as state parameters in the state set design, from which the respective objective functions were defined. To further enhance the algorithm's performance, techniques such as behavior cloning, residual networks, and CPU-GPU dual-core parallel processing modules were integrated. Through simulation experiments in the enhanced Imazu training environment, the practicality of the method, taking into account the effects of wind and ocean currents, was corroborated. The results demonstrate that the proposed algorithm can perform effective collision avoidance decision making in a range of vessel encounter situations, indicating its efficiency and robust generalization capabilities.

Enhancing Submersible Trajectory Prediction Using Grey Relational Analysis and Reinforcement Learning

Underwater Multi-agent Cooperative Formation Hunting Based on Deep Reinforcement Learning

Deep Interactive Reinforcement Learning for Path Following of Autonomous Underwater Vehicle

A COLREGs-Compliant Deep Reinforcement Learning Approach

Reinforcement Learning Based Obstacle Avoidance for AUV Swarm in Dynamic Ocean Environment

Predicting The Position of a Submarine Based on A Dynamic Model

Robust ASV Navigation Through Ground to Water Cross-Domain Deep Reinforcement Learning

Algorithms for dynamic control of a deep-sea mining vehicle based on deep reinforcement learning

Decision-Making for the Autonomous Navigation of Maritime Autonomous Surface Ships Based on Scene Division and Deep Reinforcement Learning

Research and Design of an Autonomous Underwater Vehicle Path Planning Method Based on Deep Reinforcement Learning

Path planning of autonomous underwater vehicle in unknown environment based on improved deep reinforcement learning

Path-Following Control of Unmanned Underwater Vehicle Based on an Improved TD3 Deep Reinforcement Learning

AI on the Water: Applying DRL to Autonomous Vessel Navigation

Dynamic robotic tracking of underwater targets using reinforcement learning

Unmanned Surface Vehicle Aided Maritime Data Collection Using Deep Reinforcement Learning

Secure and Cooperative Target Tracking Via AUV Swarm - A Reinforcement Learning Approach.

Dynamic Obstacle Avoidance for USVs Using Cross-Domain Deep Reinforcement Learning and Neural Network Model Predictive Controller

An autonomous coverage path planning algorithm for maritime search and rescue of persons-in-water based on deep reinforcement learning

Target Tracking Control of a Biomimetic Underwater Vehicle Through Deep Reinforcement Learning

A Deep Reinforcement Learning-based Decentralized Hierarchical Motion Control Strategy for Multiple Amphibious Spherical Robot Systems with Tilting Thrusters

Collision avoidance decision-making strategy for multiple USVs based on Deep Reinforcement Learning algorithm