Abstract:Reinforcement learning (RL) has shown superior performance in solving sequential decision problems. In recent years, RL is gradually being used to solve unmanned driving collision avoidance decision-making problems in complex scenarios. However, ships encounter many scenarios, and the differences in scenarios will seriously hinder the application of RL in collision avoidance at sea. Moreover, the iterative speed of trial-and-error learning for RL in multi-ship encounter scenarios is slow. To solve this problem, this study develops a novel intelligent collision avoidance algorithm based on approximate representation reinforcement learning (AR-RL) to realize the collision avoidance of maritime autonomous surface ships (MASS) in a continuous state space environment involving interactive learning capability like a crew in navigation situation. The new algorithm uses an approximate representation model to deal with the optimization of collision avoidance strategies in a dynamic target encounter situation. The model is combined with prior knowledge and International Regulations for Preventing Collisions at Sea (COLREGs) for optimal performance. This is followed by a design of an online solution to a value function approximation model based on gradient descent. This approach can solve the problem of large-scale collision avoidance policy learning in static-dynamic obstacles mixed environment. Finally, algorithm tests were constructed though two scenarios (i.e., the coastal static obstacle environment and the static-dynamic obstacles mixed environment) using Tianjin Port as an example and compared with multiple groups of algorithms. The results show that the algorithm can improve the large-scale learning efficiency of continuous state space of dynamic obstacle environment by approximate representation. At the same time, the MASS can efficiently and safely avoid obstacles enroute to reaching its target destination. It therefore makes significant contributions to ensuring safety at sea in a mixed traffic involving both manned and MASS in near future.

Cooperative Defense of Autonomous Surface Vessels with Quantity Disadvantage Using Behavior Cloning and Deep Reinforcement Learning

Underwater Multi-agent Cooperative Formation Hunting Based on Deep Reinforcement Learning

Distributional Reinforcement Learning based Integrated Decision Making and Control for Autonomous Surface Vehicles

Cooperative Target Enclosing Control for Multiple Unmanned Surface Vehicles with Unknown Dynamics and Safety Assurance

Impact of tight optical filtering on orthogonal time-frequency domain multiplexed signals in wavelength-selective switching systems

A novel intelligent collision avoidance algorithm based on deep reinforcement learning approach for USV

Digital Twin of Autonomous Surface Vessels for Safe Maritime Navigation Enabled through Predictive Modeling and Reinforcement Learning

Collision avoidance for autonomous ship using deep reinforcement learning and prior-knowledge-based approximate representation

Secure and Cooperative Target Tracking Via AUV Swarm - A Reinforcement Learning Approach.

A COLREGs-Compliant Deep Reinforcement Learning Approach

Multi-USV Dynamic Navigation and Target Capture: A Guided Multi-Agent Reinforcement Learning Approach

Evaluating Collaborative Autonomy in Opposed Environments using Maritime Capture-the-Flag Competitions

Using Deep Reinforcement Learning Methods for Autonomous Vessels in 2D Environments

Safe deep reinforcement learning-based adaptive control for USV interception mission

Dynamic Target Assignment by Unmanned Surface Vehicles Based on Reinforcement Learning

DRL-based target interception strategy design for an underactuated USV without obstacle collision

Clustering and Cooperative Guidance of Multiple Decoys for Defending a Naval Platform Against Salvo Threats

Unmanned Surface Vehicle Aided Maritime Data Collection Using Deep Reinforcement Learning

Intelligent AUV Surfacing Control in Network Attack Scenario

AI on the Water: Applying DRL to Autonomous Vessel Navigation

C3D: Cascade Control with Change Point Detection and Deep Koopman Learning for Autonomous Surface Vehicles