Abstract:Reinforcement learning (RL) has shown superior performance in solving sequential decision problems. In recent years, RL is gradually being used to solve unmanned driving collision avoidance decision-making problems in complex scenarios. However, ships encounter many scenarios, and the differences in scenarios will seriously hinder the application of RL in collision avoidance at sea. Moreover, the iterative speed of trial-and-error learning for RL in multi-ship encounter scenarios is slow. To solve this problem, this study develops a novel intelligent collision avoidance algorithm based on approximate representation reinforcement learning (AR-RL) to realize the collision avoidance of maritime autonomous surface ships (MASS) in a continuous state space environment involving interactive learning capability like a crew in navigation situation. The new algorithm uses an approximate representation model to deal with the optimization of collision avoidance strategies in a dynamic target encounter situation. The model is combined with prior knowledge and International Regulations for Preventing Collisions at Sea (COLREGs) for optimal performance. This is followed by a design of an online solution to a value function approximation model based on gradient descent. This approach can solve the problem of large-scale collision avoidance policy learning in static-dynamic obstacles mixed environment. Finally, algorithm tests were constructed though two scenarios (i.e., the coastal static obstacle environment and the static-dynamic obstacles mixed environment) using Tianjin Port as an example and compared with multiple groups of algorithms. The results show that the algorithm can improve the large-scale learning efficiency of continuous state space of dynamic obstacle environment by approximate representation. At the same time, the MASS can efficiently and safely avoid obstacles enroute to reaching its target destination. It therefore makes significant contributions to ensuring safety at sea in a mixed traffic involving both manned and MASS in near future.

Intelligent Navigation Method for Multiple Marine Autonomous Surface Ships Based on Improved PPO Algorithm

Autonomous navigation of marine surface vessel in extreme encounter situation

Intelligent Smart Marine Autonomous Surface Ship Decision System Based on Improved PPO Algorithm

Optimizing Anti-Collision Strategy for MASS: A Safe Reinforcement Learning Approach to Improve Maritime Traffic Safety

Decision-Making for the Autonomous Navigation of Maritime Autonomous Surface Ships Based on Scene Division and Deep Reinforcement Learning

Motion Plan Of Maritime Autonomous Surface Ships By Dynamic Programming For Collision Avoidance And Speed Optimization

Path Planning of Maritime Autonomous Surface Ships in Unknown Environment with Reinforcement Learning

Navigation Situation Adaptive Learning-Based Path Planning of Maritime Autonomous Surface Ships

Navigation Behavioural Decision-Making of MASS Based on Deep Reinforcement Learning and Artificial Potential Field

A hybrid RVO-MPPI approach for efficient collision avoidance for multiple autonomous underwater vehicles

Deep reinforcement learning with dynamic window approach based collision avoidance path planning for maritime autonomous surface ships

COLERGs-constrained Safe Reinforcement Learning for Realising MASS's Risk-Informed Collision Avoidance Decision Making

An Improved NSGA-II Algorithm for MASS Autonomous Collision Avoidance under COLREGs

Collaborative exploration of multiple unmanned surface vessels in complex areas based on PPO algorithm

A Path Planning Approach for Multi-AUV Systems with Concurrent Stationary Node Access and Adaptive Sampling

A novel model predictive artificial potential field based ship motion planning method considering COLREGs for complex encounter scenarios

A partially observable multi-ship collision avoidance decision-making model based on deep reinforcement learning

Neural Network Model-Based Reinforcement Learning Control for AUV 3-D Path Following

Collision avoidance for autonomous ship using deep reinforcement learning and prior-knowledge-based approximate representation

Dynamic adaptive intelligent navigation decision making method for multi-object situation in open water

Collision-avoidance Navigation Systems for Maritime Autonomous Surface Ships: A State of the Art Survey