Abstract:The International Journal of Robotics Research, Ahead of Print. Learning-based adaptive control methods hold the potential to empower autonomous agents in mitigating the impact of process variations with minimal human intervention. However, their application to autonomous underwater vehicles (AUVs) has been constrained by two main challenges: (1) the presence of unknown dynamics in the form of sea current disturbances, which cannot be modelled or measured due to limited sensor capability, particularly on smaller low-cost AUVs, and (2) the nonlinearity of AUV tasks, where the controller response at certain operating points must be excessively conservative to meet specifications at other points. Deep Reinforcement Learning (DRL) offers a solution to these challenges by training versatile neural network policies. Nevertheless, the application of DRL algorithms to AUVs has been predominantly limited to simulated environments due to their inherent high sample complexity and the distribution shift problem. This paper introduces a novel approach by combining the Maximum Entropy Deep Reinforcement Learning framework with a classic model-based control architecture to formulate an adaptive controller. In this framework, we propose a Sim-to-Real transfer strategy, incorporating a bio-inspired experience replay mechanism, an enhanced domain randomisation technique, and an evaluation protocol executed on a physical platform. Our experimental assessments demonstrate the effectiveness of this method in learning proficient policies from suboptimal simulated models of the AUV. When transferred to a real-world vehicle, the approach exhibits a control performance three times higher compared to its model-based nonadaptive but optimal counterpart.

Sim-to-real transfer of adaptive control parameters for AUV stabilisation under current disturbance

Sim-to-Real Transfer of Adaptive Control Parameters for AUV Stabilization under Current Disturbance

Learning an End-To-End Policy for AUV Control Within Just Forty Minutes Using Parallel Simulation

Neural Network Model-Based Reinforcement Learning Control for AUV 3-D Path Following

A General Motion Controller Based on Deep Reinforcement Learning for an Autonomous Underwater Vehicle with Unknown Disturbances.

Deep Reinforcement Learning for Vectored Thruster Autonomous Underwater Vehicle Control

Simultaneous Control and Guidance of an AUV Based on Soft Actor-Critic

Learning to Swim: Reinforcement Learning for 6-DOF Control of Thruster-driven Autonomous Underwater Vehicles

Reinforcement Learning for Autonomous Underwater Vehicles Via Data-Informed Domain Randomization

A Fast Adaptive AUV Control Policy Based on Progressive Networks with Context Information

Intelligent AUV Surfacing Control in Network Attack Scenario

Deep Reinforcement Learning Based Tracking Control of an Autonomous Surface Vessel in Natural Waters

Attention-Based Meta-Reinforcement Learning for Tracking Control of AUV with Time-Varying Dynamics

End-To-End Sensorimotor Control Problems Of Auvs With Deep Reinforcement Learning

AUV Obstacle Avoidance Framework Based on Event-Triggered Reinforcement Learning

Deep reinforcement learning for adaptive path planning and control of an autonomous underwater vehicle

Single degree of freedom control based on deep reinforcement learning for underwater unmanned vehicle

Adversarial Deep Reinforcement Learning Based Robust Depth Tracking Control for Underactuated Autonomous Underwater Vehicle

AUV Path Following Control using Deep Reinforcement Learning Under the Influence of Ocean Currents.

General reinforcement learning control for AUV manoeuvring in turbulent flows

Exploration of the Applicability of Probabilistic Inference for Learning Control in Underactuated Autonomous Underwater Vehicles