Abstract:The International Journal of Robotics Research, Ahead of Print. Learning-based adaptive control methods hold the potential to empower autonomous agents in mitigating the impact of process variations with minimal human intervention. However, their application to autonomous underwater vehicles (AUVs) has been constrained by two main challenges: (1) the presence of unknown dynamics in the form of sea current disturbances, which cannot be modelled or measured due to limited sensor capability, particularly on smaller low-cost AUVs, and (2) the nonlinearity of AUV tasks, where the controller response at certain operating points must be excessively conservative to meet specifications at other points. Deep Reinforcement Learning (DRL) offers a solution to these challenges by training versatile neural network policies. Nevertheless, the application of DRL algorithms to AUVs has been predominantly limited to simulated environments due to their inherent high sample complexity and the distribution shift problem. This paper introduces a novel approach by combining the Maximum Entropy Deep Reinforcement Learning framework with a classic model-based control architecture to formulate an adaptive controller. In this framework, we propose a Sim-to-Real transfer strategy, incorporating a bio-inspired experience replay mechanism, an enhanced domain randomisation technique, and an evaluation protocol executed on a physical platform. Our experimental assessments demonstrate the effectiveness of this method in learning proficient policies from suboptimal simulated models of the AUV. When transferred to a real-world vehicle, the approach exhibits a control performance three times higher compared to its model-based nonadaptive but optimal counterpart.

General reinforcement learning control for AUV manoeuvring in turbulent flows

Neural Network Model-Based Reinforcement Learning Control for AUV 3-D Path Following

A General Motion Controller Based on Deep Reinforcement Learning for an Autonomous Underwater Vehicle with Unknown Disturbances.

Sim-to-Real Transfer of Adaptive Control Parameters for AUV Stabilization under Current Disturbance

A Learning-Based Flexible Autonomous Motion Control Method for UAV in Dynamic Unknown Environments

Deep Reinforcement Learning for Vectored Thruster Autonomous Underwater Vehicle Control

Applying Deep Reinforcement Learning To Active Flow Control In Weakly Turbulent Conditions

Sim-to-real transfer of adaptive control parameters for AUV stabilisation under current disturbance

AUV Path Following Control using Deep Reinforcement Learning Under the Influence of Ocean Currents.

Reinforcement Learning for Autonomous Underwater Vehicles Via Data-Informed Domain Randomization

AUV Obstacle Avoidance Framework Based on Event-Triggered Reinforcement Learning

Intelligent AUV Surfacing Control in Network Attack Scenario

2-Level Reinforcement Learning for Ships on Inland Waterways: Path Planning and Following

Learning to Swim: Reinforcement Learning for 6-DOF Control of Thruster-driven Autonomous Underwater Vehicles

Deep Reinforcement Learning Based Tracking Control of an Autonomous Surface Vessel in Natural Waters

AUV position tracking and trajectory control based on fast-deployed deep reinforcement learning method

Depth Control of Model-Free AUVs Via Reinforcement Learning

Attention-Based Meta-Reinforcement Learning for Tracking Control of AUV with Time-Varying Dynamics

Path-Following Control of Unmanned Underwater Vehicle Based on an Improved TD3 Deep Reinforcement Learning

A selected review on reinforcement learning based control for autonomous underwater vehicles

Reinforcement Learning Based Obstacle Avoidance for Autonomous Underwater Vehicle