Abstract:Reinforcement learning (RL) has shown superior performance in solving sequential decision problems. In recent years, RL is gradually being used to solve unmanned driving collision avoidance decision-making problems in complex scenarios. However, ships encounter many scenarios, and the differences in scenarios will seriously hinder the application of RL in collision avoidance at sea. Moreover, the iterative speed of trial-and-error learning for RL in multi-ship encounter scenarios is slow. To solve this problem, this study develops a novel intelligent collision avoidance algorithm based on approximate representation reinforcement learning (AR-RL) to realize the collision avoidance of maritime autonomous surface ships (MASS) in a continuous state space environment involving interactive learning capability like a crew in navigation situation. The new algorithm uses an approximate representation model to deal with the optimization of collision avoidance strategies in a dynamic target encounter situation. The model is combined with prior knowledge and International Regulations for Preventing Collisions at Sea (COLREGs) for optimal performance. This is followed by a design of an online solution to a value function approximation model based on gradient descent. This approach can solve the problem of large-scale collision avoidance policy learning in static-dynamic obstacles mixed environment. Finally, algorithm tests were constructed though two scenarios (i.e., the coastal static obstacle environment and the static-dynamic obstacles mixed environment) using Tianjin Port as an example and compared with multiple groups of algorithms. The results show that the algorithm can improve the large-scale learning efficiency of continuous state space of dynamic obstacle environment by approximate representation. At the same time, the MASS can efficiently and safely avoid obstacles enroute to reaching its target destination. It therefore makes significant contributions to ensuring safety at sea in a mixed traffic involving both manned and MASS in near future.

Autonomous Surface Vessel Obstacle Avoidance Based on Hierarchical Reinforcement Learning

Robust ASV Navigation Through Ground to Water Cross-Domain Deep Reinforcement Learning

Reinforcement Learning Based Obstacle Avoidance for Autonomous Underwater Vehicle

Neural Network Model-Based Reinforcement Learning Control for AUV 3-D Path Following

AUV Obstacle Avoidance Framework Based on Event-Triggered Reinforcement Learning

Surface path tracking method of autonomous surface underwater vehicle based on deep reinforcement learning

AI on the Water: Applying DRL to Autonomous Vessel Navigation

Research on obstacle avoidance of underactuated autonomous underwater vehicle based on offline reinforcement learning

Concise Deep Reinforcement Learning Obstacle Avoidance for Underactuated Unmanned Marine Vessels.

Collision avoidance for autonomous ship using deep reinforcement learning and prior-knowledge-based approximate representation

Method for Intelligent Obstacle Avoidance Decision-Making of Unmanned Vessel in Unknown Waters

Obstacle avoidance USV in multi-static obstacle environments based on a deep reinforcement learning approach

Dynamic Obstacle Avoidance for USVs Using Cross-Domain Deep Reinforcement Learning and Neural Network Model Predictive Controller

Decision-Making for the Autonomous Navigation of Maritime Autonomous Surface Ships Based on Scene Division and Deep Reinforcement Learning

A COLREGs-Compliant Deep Reinforcement Learning Approach

Decentralized Multi-Robot Navigation for Autonomous Surface Vehicles with Distributional Reinforcement Learning

Obstacle avoidance for environmentally-driven USVs based on deep reinforcement learning in large-scale uncertain environments

Autonomous obstacle avoidance of UAV based on deep reinforcement learning

Reinforcement Learning Based Obstacle Avoidance for AUV Swarm in Dynamic Ocean Environment

Research on collision avoidance algorithm of unmanned surface vehicle based on deep reinforcement learning

Deep Interactive Reinforcement Learning for Path Following of Autonomous Underwater Vehicle