Abstract:Automatic collision avoidance decision making for vessels is a critical challenge in the development of autonomous ships and has become a central point of research in the maritime safety domain. Effective and systematic collision avoidance strategies significantly reduce the risk of vessel collisions, ensuring safe navigation. This study develops a multi-vessel automatic collision avoidance decision-making method based on deep reinforcement learning (DRL) and establishes a vessel behavior decision model. When designing the reward function for continuous action spaces, the criteria of the "Convention on the International Regulations for Preventing Collisions at Sea" (COLREGs) were adhered to, taking into account the vessel's collision risk under various encounter situations, real-world navigation practices, and navigational complexities. Furthermore, to enable the algorithm to precisely differentiate between collision avoidance and the navigation resumption phase in varied vessel encounter situations, this paper incorporated "collision avoidance decision making" and "course recovery decision making" as state parameters in the state set design, from which the respective objective functions were defined. To further enhance the algorithm's performance, techniques such as behavior cloning, residual networks, and CPU-GPU dual-core parallel processing modules were integrated. Through simulation experiments in the enhanced Imazu training environment, the practicality of the method, taking into account the effects of wind and ocean currents, was corroborated. The results demonstrate that the proposed algorithm can perform effective collision avoidance decision making in a range of vessel encounter situations, indicating its efficiency and robust generalization capabilities.

An Aircraft Collision Avoidance Method Based on Deep Reinforcement Learning

A Method for Pilot Acceptability Evaluation of Airborne Collision Avoidance System Based on Genetic Algorithm

Optimizing Collision Avoidance in Dense Airspace using Deep Reinforcement Learning

Reward Function Optimization of a Deep Reinforcement Learning Collision Avoidance System

Deep Reinforcement Learning for Advanced Longitudinal Control and Collision Avoidance in High-Risk Driving Scenarios

Autonomous air traffic separation assurance through machine learning

Deep Reinforcement Learning Based Conflict Detection and Resolution in Air Traffic Control

A Learning Method for AUV Collision Avoidance Through Deep Reinforcement Learning

Graphical Modeling and Simulation for a Multi-Aircraft Collision Avoidance Algorithm Based on Collaborative Decisions.

Autonomous Air Traffic Controller: A Deep Multi-Agent Reinforcement Learning Approach

Learning Configurations of Operating Environment of Autonomous Vehicles to Maximize Their Collisions

Formally Verified Next-Generation Airborne Collision Avoidance Games in ACAS X

Research on Unmanned Surface Vehicle Collision Avoidance Based on Deep Reinforcement Learning

Over-the-Horizon Air Combat Environment Modeling and Deep Reinforcement Learning Application

An Autonomous Free Airspace En-route Controller using Deep Reinforcement Learning Techniques

Deep Reinforcement-Learning-Based Air-Combat-Maneuver Generation Framework

Reinforcement Learning-Based Collision Avoidance Guidance Algorithm for Fixed-Wing UAVs.

A COLREGs-Compliant Deep Reinforcement Learning Approach

A safe reinforcement learning approach for autonomous navigation of mobile robots in dynamic environments

Deep-Reinforcement-Learning-Based Collision Avoidance of Autonomous Driving System for Vulnerable Road User Safety

Dynamic Obstacle Avoidance of Fixed-wing Aircraft in Final Phase Via Reinforcement Learning