Abstract:Reinforcement learning (RL) has shown superior performance in solving sequential decision problems. In recent years, RL is gradually being used to solve unmanned driving collision avoidance decision-making problems in complex scenarios. However, ships encounter many scenarios, and the differences in scenarios will seriously hinder the application of RL in collision avoidance at sea. Moreover, the iterative speed of trial-and-error learning for RL in multi-ship encounter scenarios is slow. To solve this problem, this study develops a novel intelligent collision avoidance algorithm based on approximate representation reinforcement learning (AR-RL) to realize the collision avoidance of maritime autonomous surface ships (MASS) in a continuous state space environment involving interactive learning capability like a crew in navigation situation. The new algorithm uses an approximate representation model to deal with the optimization of collision avoidance strategies in a dynamic target encounter situation. The model is combined with prior knowledge and International Regulations for Preventing Collisions at Sea (COLREGs) for optimal performance. This is followed by a design of an online solution to a value function approximation model based on gradient descent. This approach can solve the problem of large-scale collision avoidance policy learning in static-dynamic obstacles mixed environment. Finally, algorithm tests were constructed though two scenarios (i.e., the coastal static obstacle environment and the static-dynamic obstacles mixed environment) using Tianjin Port as an example and compared with multiple groups of algorithms. The results show that the algorithm can improve the large-scale learning efficiency of continuous state space of dynamic obstacle environment by approximate representation. At the same time, the MASS can efficiently and safely avoid obstacles enroute to reaching its target destination. It therefore makes significant contributions to ensuring safety at sea in a mixed traffic involving both manned and MASS in near future.

Case learning for CBR-based collision avoidance systems

Case Learning in CBR-Based Agent Systems for Ship Collision Avoidance

CBR Based Collision Avoidance Approach to Reactive Control for Multi-Robot System

Using cased based reasoning for automated safety risk management in construction industry

Developing A Case-Based Reasoning Model for Safety Accident Pre-Control and Decision Making in the Construction Industry

A CBR-Based Multiagent Reinforcement Learning Approach

Research of new strategies for improving CBR system

Research on CBR System Based on Data Mining

An Open Case-based Reasoning Framework for Personalized On-board Driving Assistance in Risk Scenarios

Knowledge transfer enabled reinforcement learning for efficient and safe autonomous ship collision avoidance

A Systematic Approach for Collision Risk Analysis Based on Ais Data

Apply Case-Based Reasoning to Collaborative Commerce

Representing and matching simulation cases: A case-based reasoning approach

Efficient Reinforcement Learning for Autonomous Ship Collision Avoidance under Learning Experience Reuse

A Hybrid Retrieval Strategy for Case-based Reasoning Using Soft Likelihood Functions

Reinforcement learning-based collision avoidance: impact of reward function and knowledge transfer

Knowledge graph and CBR-based approach for automated analysis of bridge operational accidents: Case representation and retrieval

Research and implementation of a multi-fault diagnosis method based on case-based reasoning

Collision avoidance for autonomous ship using deep reinforcement learning and prior-knowledge-based approximate representation

Intelligent Q&A System Based on Case Based Reasoning