Abstract:The International Journal of Robotics Research, Ahead of Print. Learning-based adaptive control methods hold the potential to empower autonomous agents in mitigating the impact of process variations with minimal human intervention. However, their application to autonomous underwater vehicles (AUVs) has been constrained by two main challenges: (1) the presence of unknown dynamics in the form of sea current disturbances, which cannot be modelled or measured due to limited sensor capability, particularly on smaller low-cost AUVs, and (2) the nonlinearity of AUV tasks, where the controller response at certain operating points must be excessively conservative to meet specifications at other points. Deep Reinforcement Learning (DRL) offers a solution to these challenges by training versatile neural network policies. Nevertheless, the application of DRL algorithms to AUVs has been predominantly limited to simulated environments due to their inherent high sample complexity and the distribution shift problem. This paper introduces a novel approach by combining the Maximum Entropy Deep Reinforcement Learning framework with a classic model-based control architecture to formulate an adaptive controller. In this framework, we propose a Sim-to-Real transfer strategy, incorporating a bio-inspired experience replay mechanism, an enhanced domain randomisation technique, and an evaluation protocol executed on a physical platform. Our experimental assessments demonstrate the effectiveness of this method in learning proficient policies from suboptimal simulated models of the AUV. When transferred to a real-world vehicle, the approach exhibits a control performance three times higher compared to its model-based nonadaptive but optimal counterpart.

Data-Driven Performance-Prescribed Reinforcement Learning Control of an Unmanned Surface Vehicle

A Model Predictive Control Approach for USV Autonomous Cruising Via Disturbance Learning

Event-Triggered Optimal Tracking Control for Underactuated Surface Vessels Via Neural Reinforcement Learning

Neural Network Model-Based Reinforcement Learning Control for AUV 3-D Path Following

Path Following Control for Unmanned Surface Vehicles: A Reinforcement Learning-Based Method with Experimental Validation.

Performance-Guaranteed Adaptive Optimized Control of Intelligent Surface Vehicle Using Reinforcement Learning

Adaptive Optimal Tracking Control for Uncertain Unmanned Surface Vessel Via Reinforcement Learning

LSTM-DPPO based deep reinforcement learning controller for path following optimization of unmanned surface vehicle

Adaptive Neural Network Iterative Sliding Mode Course Tracking Control for Unmanned Surface Vessels

Adaptive Sliding Mode Control Design for Nonlinear Unmanned Surface Vessel Using RBFNN and Disturbance-Observer

USV Trajectory Tracking Control Based on Receding Horizon Reinforcement Learning

Collision Avoidance and Path Point Tracking Control for Underactuated Unmanned Surface Vehicles with Unknown Model Nonlinearity

Reinforcement learning-based finite-time tracking control of an unknown unmanned surface vehicle with input constraints

Self‐learning‐based optimal tracking control of an unmanned surface vehicle with pose and velocity constraints

Reinforcement Learning-Based Tracking Control of USVs in Varying Operational Conditions

Single degree of freedom control based on deep reinforcement learning for underwater unmanned vehicle

End-to-End Trajectory Tracking Algorithm for Unmanned Surface Vehicle Using Reinforcement Learning

An Offline Reinforcement Learning Approach for Path Following of an Unmanned Surface Vehicle

Intelligent Controller for Unmanned Surface Vehicles by Deep Reinforcement Learning

Sim-to-Real Transfer of Adaptive Control Parameters for AUV Stabilization under Current Disturbance

Sim-to-real transfer of adaptive control parameters for AUV stabilisation under current disturbance