Abstract:In this article, we present an adaptive reinforcement learning optimal tracking control (RLOTC) algorithm for an underactuated surface vessel subject to modeling uncertainties and time-varying external disturbances. By integrating backstepping technique with the optimized control design, we show that the desired optimal tracking performance of vessel control is guaranteed due to the fact that the virtual and actual control inputs are designed as optimized solutions of every subsystem. To enhance the robustness of vessel control systems, we employ neural network (NN) approximators to approximate uncertain vessel dynamics and present adaptive control technique to estimate the upper boundedness of external disturbances. Under the reinforcement learning framework, we construct actor-critic networks to solve the Hamilton-Jacobi-Bellman equations corresponding to subsystems of surface vessel to achieve the optimized control. The optimized control algorithm can synchronously train the adaptive parameters not only for actor-critic networks but also for NN approximators and adaptive control. By Lyapunov stability theorem, we show that the RLOTC algorithm can ensure the semiglobal uniform ultimate boundedness of the closed-loop systems. Compared with the existing reinforcement learning control results, the presented RLOTC algorithm can compensate for uncertain vessel dynamics and unknown disturbances, and obtain the optimized control performance by considering optimization in every backstepping design. Simulation studies on an underactuated surface vessel are given to illustrate the effectiveness of the RLOTC algorithm.

Online Approximate Optimal Station Keeping of an Autonomous Underwater Vehicle

Position Tracking Control of Fully-Actuated Underwater Vehicles with Constrained Attitude and Velocities

Finite-time Containment Control for Autonomous Underwater Vehicles with Prescribed Performance

A Model Predictive Control Approach for USV Autonomous Cruising Via Disturbance Learning

Human-in-the-loop Distributed Cooperative Tracking Control with Applications to Autonomous Ground Vehicles: A Data-Driven Mixed Iteration Approach

Learning-based Robust Optimal Tracking Controller Design for Unmanned Underwater Vehicles with Full-State and Input Constraints

Adaptive Reinforcement Learning Strategy-Based Sliding Mode Control of Uncertain Euler-Lagrange Systems with Prescribed Performance Guarantees: Autonomous Underwater Vehicles-Based Verification

Distributed Neurodynamics-Based Backstepping Optimal Control for Robust Constrained Consensus of Underactuated Underwater Vehicles Fleet

Model-free control of underwater vehicle-manipulator system interacting with unknown environments

Adaptive Optimal Tracking Control of an Underactuated Surface Vessel Using Actor–Critic Reinforcement Learning

High-Level Tracking of Autonomous Underwater Vehicles Based on Pseudo Averaged Q-Learning.

Three-dimensional optimal trajectory tracking control of underactuated AUVs with uncertain dynamics and input saturation

An optimal consensus tracking control algorithm for autonomous underwater vehicles with disturbances

Hybrid offline-online reinforcement learning for obstacle avoidance in autonomous underwater vehicles

Learning to Swim: Reinforcement Learning for 6-DOF Control of Thruster-driven Autonomous Underwater Vehicles

Self‐learning‐based optimal tracking control of an unmanned surface vehicle with pose and velocity constraints

Adaptive Optimal Surrounding Control of Multiple Unmanned Surface Vessels via Actor-Critic Reinforcement Learning

Optimal course tracking control of USV with input dead zone based on adaptive fuzzy dynamic programing

Adaptive Formation Learning Control for Cooperative AUVs under Complete Uncertainty

Station-Keeping Control of a Hovering Over-Actuated Autonomous Underwater Vehicle Under Ocean Current Effects and Model Uncertainties in Horizontal Plane

A Robust Model Predictive Control Approach for Autonomous Underwater Vehicles Operating in a Constrained workspace