Abstract:Abstract Collision avoidance in ships and robotic vehicles exemplifies a complex work process that necessitates effective scenario recognition and precise movement decision-making. Machine learning methods addressing such work processes generally involve learning from scratch, which is not only time-consuming but also demands significant computational resources. Transfer learning emerges as a potent strategy to enhance the efficiency of these engineering work processes by harnessing previously acquired knowledge from analogous tasks, thereby streamlining the learning curve for new challenges. This research delves into two critical questions central to optimizing transfer reinforcement learning for the work process of collision avoidance: (1) Which process features can be successfully transferred across varying work processes? (2) What methodologies support the efficient and effective transfer of these features? Our study employs simulation-based experiments in ship collision avoidance to address these questions, chosen for their intrinsic complexity and the varied feature recognition it demands. We investigate and compare two transfer learning techniques—feature extraction and finetuning—utilizing a lightweight convolutional neural network (CNN) model pretrained on a base case of a comparable work process. Pixel-level visual input is leveraged to cover different numbers of encountering ships and fix the input size for the model. This model adeptly demonstrates the feasibility of transferring essential features to newer work process scenarios. Further, to enhance realism and applicability, we introduce a simplified yet comprehensive ship dynamic model that considers the substantial effects of ship inertia, thereby refining the interaction between the model and its environment. The response time is embedded into the reward function design to be considered for policy training. Experimental outcomes underscore the transferability of diverse process features and evaluate the relative effectiveness of the employed transfer methods across different task settings, offering insights that could be extrapolated to other engineering work processes.

Efficient Reinforcement Learning for Autonomous Ship Collision Avoidance under Learning Experience Reuse

Collision avoidance for autonomous ship using deep reinforcement learning and prior-knowledge-based approximate representation

Knowledge transfer enabled reinforcement learning for efficient and safe autonomous ship collision avoidance

A human-like collision avoidance method for autonomous ship with attention-based deep reinforcement learning

A COLREGs-Compliant Deep Reinforcement Learning Approach

Reinforcement learning-based collision avoidance: impact of reward function and knowledge transfer

Adaptive Collision Avoidance Decisions in Autonomous Ship Encounter Scenarios Through Rule-Guided Vision Supervised Learning

Enhancing Efficiency in Collision Avoidance: A Study on Transfer Reinforcement Learning in Autonomous Ships’ Navigation

AUV Collision Avoidance Strategy Based on Fuzzy Reinforcement Learning

An Autonomous Decision-making Algorithm for Ship Collision Avoidance Based on DDQN with Prioritized Experience Replay

A Novel Reinforcement Learning Collision Avoidance Algorithm for USVs Based on Maneuvering Characteristics and COLREGs

Deep Reinforcement Learning Based Path Planning and Collision Avoidance for Smart Ships in Complex Environments

Collision avoidance decision-making strategy for multiple USVs based on Deep Reinforcement Learning algorithm

COLERGs-constrained Safe Reinforcement Learning for Realising MASS's Risk-Informed Collision Avoidance Decision Making

SSRL: A Safe and Smooth Reinforcement Learning Approach for Collision Avoidance in Navigation

A Learning Method for AUV Collision Avoidance Through Deep Reinforcement Learning

Research on collision avoidance algorithm of unmanned surface vehicle based on deep reinforcement learning

Improved reinforcement learning for collision-free local path planning of dynamic obstacle

A Novel Deep Reinforcement Learning for POMDP-based Autonomous Ship Collision Decision-Making

Adaptive Environment Modeling Based Reinforcement Learning for Collision Avoidance in Complex Scenes