Abstract:Automatic collision avoidance decision making for vessels is a critical challenge in the development of autonomous ships and has become a central point of research in the maritime safety domain. Effective and systematic collision avoidance strategies significantly reduce the risk of vessel collisions, ensuring safe navigation. This study develops a multi-vessel automatic collision avoidance decision-making method based on deep reinforcement learning (DRL) and establishes a vessel behavior decision model. When designing the reward function for continuous action spaces, the criteria of the "Convention on the International Regulations for Preventing Collisions at Sea" (COLREGs) were adhered to, taking into account the vessel's collision risk under various encounter situations, real-world navigation practices, and navigational complexities. Furthermore, to enable the algorithm to precisely differentiate between collision avoidance and the navigation resumption phase in varied vessel encounter situations, this paper incorporated "collision avoidance decision making" and "course recovery decision making" as state parameters in the state set design, from which the respective objective functions were defined. To further enhance the algorithm's performance, techniques such as behavior cloning, residual networks, and CPU-GPU dual-core parallel processing modules were integrated. Through simulation experiments in the enhanced Imazu training environment, the practicality of the method, taking into account the effects of wind and ocean currents, was corroborated. The results demonstrate that the proposed algorithm can perform effective collision avoidance decision making in a range of vessel encounter situations, indicating its efficiency and robust generalization capabilities.

An Active-Controlled Heaving Plate Breakwater Trained by an Intelligent Framework Based on Deep Reinforcement Learning

Control of a Wave Energy Converter Using Model-Free Deep Reinforcement Learning

Environmental-Sensing and adaptive optimization of wave energy converter based on deep reinforcement learning and computational fluid dynamics

A Simple Approach for Wave Absorbing Control of Plunger Wavemakers Using Machine Learning: Numerical Study

Numerical Study on the Wave Dissipating Performance of a Submerged Heaving Plate Breakwater

Sloshing Suppression with Active Controlled Baffles Through Deep Reinforcement Learning–expert Demonstrations–behavior Cloning Process

Deep Reinforcement Learning: A New Beacon for Intelligent Active Flow Control

Design and Tank Testing of Reinforcement Learning Control for Wave Energy Converters

Active flow control with rotating cylinders by an artificial neural network trained by deep reinforcement learning

Deep Reinforcement Learning Based Controller for Active Heave Compensation

Efficient Active Flow Control Strategy for Confined Square Cylinder Wake Using Deep Learning-Based Surrogate Model and Reinforcement Learning

Bluff body uses deep-reinforcement-learning trained active flow control to achieve hydrodynamic stealth

Adaptive control of transonic buffet and buffeting flow with deep reinforcement learning

A field programmable gate array-based deep reinforcement learning framework for experimental active flow control and its application in airfoil flow separation elimination

DRLinFluids---An open-source python platform of coupling Deep Reinforcement Learning and OpenFOAM

Reinforcement-learning-based Parameter Optimization of a Splitter Plate Downstream in Cylinder Wake with Stability Analyses

2-Level Reinforcement Learning for Ships on Inland Waterways: Path Planning and Following

Application of Deep Reinforcement Learning Control of an Inverted Hydraulic Pendulum

Deep Reinforcement Learning Controller for 3D Path Following and Collision Avoidance by Autonomous Underwater Vehicles

A COLREGs-Compliant Deep Reinforcement Learning Approach

How to Control Hydrodynamic Force on Fluidic Pinball via Deep Reinforcement Learning