Abstract:The research on decision-making models of ship collision avoidance is confronted with numerous challenges. These challenges encompass inadequate consideration of complex factors, including but not limited to open water scenarios, the absence of static obstacle considerations, and insufficient attention given to avoiding collisions between manned ships and MASSs. A decision model for MASS collision avoidance is proposed to overcome these limitations by integrating the strengths of model-based and model-free methods in reinforcement learning. This model incorporates S-57 chart information, AIS data, and the Dyna framework to improve effectiveness. (1) When the MASS's navigation task is known, a static navigation environment is built based on S-57 chart information, and the Voronoi diagram and improved A* algorithm are used to obtain the energy-saving optimal static path as the planned sea route. (2) Given the small main dimensions of an MASS, which is easily affected by wind and current factors, the motion model of an MASS is established based on the MMG model considering wind and current factors. At the same time, AIS data are used to extract the target ship (manned ship) data. (3) According to the characteristics of the actual navigation of ships at sea, the state space, action space, and reward function of the reinforcement learning algorithm are designed. The MASS collision avoidance decision model based on the Dyna-DQN model is established. Based on the DQN algorithm, the agent (MASS) and the environment interact continuously, and the actual interaction data generated are used for the iterative update of the collision avoidance strategy and the training of the environment model. Then, the environment model is used to generate a series of simulated empirical data to promote the iterative update of the strategy. Using the waters near the South China Sea as the research object for simulation verification, the navigation tasks are divided into three categories: only considering static obstacles, following the planned sea route considering static obstacles, and following the planned sea route considering both static and dynamic obstacles. The results show that through repeated simulation experiments, an MASS can complete the navigation task without colliding with static and dynamic obstacles. Therefore, the proposed method can be used in the intelligent collision avoidance module of MASSs and is an effective MASS collision avoidance method.

Method for Intelligent Obstacle Avoidance Decision-Making of Unmanned Vessel in Unknown Waters

Research on Intelligent Collision Avoidance Decision-Making of Unmanned Ship in Unknown Environments

Optimizing Multi-Vessel Collision Avoidance Decision Making for Autonomous Surface Vessels: A COLREGs-Compliant Deep Reinforcement Learning Approach

Decision-making for the Autonomous Navigation of USVs Based on Deep Reinforcement Learning under IALA Maritime Buoyage System

A COLREGs-Compliant Deep Reinforcement Learning Approach

Intelligent collision avoidance decision-making method for unmanned ships based on driving practice

Decision-Making for the Autonomous Navigation of Maritime Autonomous Surface Ships Based on Scene Division and Deep Reinforcement Learning

Collision Avoidance and Path Point Tracking Control for Underactuated Unmanned Surface Vehicles with Unknown Model Nonlinearity

An intelligent collision avoidance and navigation approach of unmanned surface vessel considering navigation experience and rules

Intelligent Decision-Making for 3-Dimensional Dynamic Obstacle Avoidance of UAV Based on Deep Reinforcement Learning

Research on Unmanned Surface Vehicle Collision Avoidance Based on Deep Reinforcement Learning

AI on the Water: Applying DRL to Autonomous Vessel Navigation

Obstacle avoidance USV in multi-static obstacle environments based on a deep reinforcement learning approach

A human-like collision avoidance method for USVs based on deep reinforcement learning and velocity obstacle

Research on collision avoidance algorithm of unmanned surface vehicle based on deep reinforcement learning

Dynamic adaptive intelligent navigation decision making method for multi-object situation in open water

A Multi-Stage Collision Avoidance Model for Autonomous Ship Based on Fuzzy Set Theory with TL-DDQN Algorithm

Navigation Behavioural Decision-Making of MASS Based on Deep Reinforcement Learning and Artificial Potential Field

Autonomous Collision Avoidance Decision-Making Method for USV Based on ATL-TD3 Algorithm

Unmanned Ship Collision Avoidance Action Plan Deduction Method under Man–Machine Interactive Negotiation in Collision Avoidance Scenarios

Research on MASS Collision Avoidance in Complex Waters Based on Deep Reinforcement Learning