Abstract:Abstract Collision avoidance in ships and robotic vehicles exemplifies a complex work process that necessitates effective scenario recognition and precise movement decision-making. Machine learning methods addressing such work processes generally involve learning from scratch, which is not only time-consuming but also demands significant computational resources. Transfer learning emerges as a potent strategy to enhance the efficiency of these engineering work processes by harnessing previously acquired knowledge from analogous tasks, thereby streamlining the learning curve for new challenges. This research delves into two critical questions central to optimizing transfer reinforcement learning for the work process of collision avoidance: (1) Which process features can be successfully transferred across varying work processes? (2) What methodologies support the efficient and effective transfer of these features? Our study employs simulation-based experiments in ship collision avoidance to address these questions, chosen for their intrinsic complexity and the varied feature recognition it demands. We investigate and compare two transfer learning techniques—feature extraction and finetuning—utilizing a lightweight convolutional neural network (CNN) model pretrained on a base case of a comparable work process. Pixel-level visual input is leveraged to cover different numbers of encountering ships and fix the input size for the model. This model adeptly demonstrates the feasibility of transferring essential features to newer work process scenarios. Further, to enhance realism and applicability, we introduce a simplified yet comprehensive ship dynamic model that considers the substantial effects of ship inertia, thereby refining the interaction between the model and its environment. The response time is embedded into the reward function design to be considered for policy training. Experimental outcomes underscore the transferability of diverse process features and evaluate the relative effectiveness of the employed transfer methods across different task settings, offering insights that could be extrapolated to other engineering work processes.

Adaptive Collision Avoidance Decisions in Autonomous Ship Encounter Scenarios Through Rule-Guided Vision Supervised Learning

A COLREGs-Compliant Deep Reinforcement Learning Approach

Ship collision risk analysis: Modeling, visualization and prediction

Collision avoidance for autonomous ship using deep reinforcement learning and prior-knowledge-based approximate representation

A human-like collision avoidance method for autonomous ship with attention-based deep reinforcement learning

Research on collision avoidance algorithm of unmanned surface vehicle based on deep reinforcement learning

A human-like collision avoidance method for USVs based on deep reinforcement learning and velocity obstacle

Knowledge transfer enabled reinforcement learning for efficient and safe autonomous ship collision avoidance

Collision avoidance decision-making strategy for multiple USVs based on Deep Reinforcement Learning algorithm

Enhancing Efficiency in Collision Avoidance: A Study on Transfer Reinforcement Learning in Autonomous Ships’ Navigation

Decision-Making for the Autonomous Navigation of Maritime Autonomous Surface Ships Based on Scene Division and Deep Reinforcement Learning

Efficient Reinforcement Learning for Autonomous Ship Collision Avoidance under Learning Experience Reuse

Deep Reinforcement Learning Based Path Planning and Collision Avoidance for Smart Ships in Complex Environments

A path planning strategy unified with a COLREGS collision avoidance function based on deep reinforcement learning and artificial potential field

AI on the Water: Applying DRL to Autonomous Vessel Navigation

A Novel Reinforcement Learning Collision Avoidance Algorithm for USVs Based on Maneuvering Characteristics and COLREGs

A Novel Deep Reinforcement Learning for POMDP-based Autonomous Ship Collision Decision-Making

A novel intelligent collision avoidance algorithm based on deep reinforcement learning approach for USV

Automatic ship collision avoidance using deep reinforcement learning with LSTM in continuous action spaces

COLERGs-constrained Safe Reinforcement Learning for Realising MASS's Risk-Informed Collision Avoidance Decision Making

An Autonomous Decision-making Algorithm for Ship Collision Avoidance Based on DDQN with Prioritized Experience Replay