Abstract:In this article, we present an adaptive reinforcement learning optimal tracking control (RLOTC) algorithm for an underactuated surface vessel subject to modeling uncertainties and time-varying external disturbances. By integrating backstepping technique with the optimized control design, we show that the desired optimal tracking performance of vessel control is guaranteed due to the fact that the virtual and actual control inputs are designed as optimized solutions of every subsystem. To enhance the robustness of vessel control systems, we employ neural network (NN) approximators to approximate uncertain vessel dynamics and present adaptive control technique to estimate the upper boundedness of external disturbances. Under the reinforcement learning framework, we construct actor-critic networks to solve the Hamilton-Jacobi-Bellman equations corresponding to subsystems of surface vessel to achieve the optimized control. The optimized control algorithm can synchronously train the adaptive parameters not only for actor-critic networks but also for NN approximators and adaptive control. By Lyapunov stability theorem, we show that the RLOTC algorithm can ensure the semiglobal uniform ultimate boundedness of the closed-loop systems. Compared with the existing reinforcement learning control results, the presented RLOTC algorithm can compensate for uncertain vessel dynamics and unknown disturbances, and obtain the optimized control performance by considering optimization in every backstepping design. Simulation studies on an underactuated surface vessel are given to illustrate the effectiveness of the RLOTC algorithm.

Adaptive Critic Learning for Optimized Tracking Control of an Unmanned Surface Vehicle with Guaranteed Performance

Robust Trajectory Tracking Control of Underactuated Unmanned Surface Vehicles with Exponential Stability: Theory and Experimental Validation.

Performance-Guaranteed Adaptive Optimized Control of Intelligent Surface Vehicle Using Reinforcement Learning

Performance-Guaranteed Tracking Control of an Autonomous Surface Vessel with Parametric Uncertainties and Time-Varying Disturbances.

Self‐learning‐based optimal tracking control of an unmanned surface vehicle with pose and velocity constraints

Adaptive Neural Optimal Tracking Control for Uncertain Unmanned Surface Vehicle

State constrained control strategy for unmanned surface vehicle trajectory tracking based on improved barrier Lyapunov function

Online Reinforcement Learning-Based Adaptive Tracking Control of an Unknown Unmanned Surface Vehicle with Input Saturations

Fuzzy Optimal Tracking Control for Unmanned Surface Vehicles with Prescribed-Time Convergence Analysis

Fuzzy Optimal Tracking Control for Nonlinear Underactuated Unmanned Surface Vehicles

Adaptive Optimal Tracking Control of an Underactuated Surface Vessel Using Actor–Critic Reinforcement Learning

Adaptive-critic-design based tracking controller for boost complement glide vehicle with performance constraint and input saturation

Robust Adaptive Optimal Trajectory Tracking Control for Underactuated AUVs with Position and Velocity Constraints in Three‐dimensional Space

Reinforcement Learning-Based Optimal Tracking Control of an Unknown Unmanned Surface Vehicle.

Adaptive Optimal Tracking Control for Uncertain Unmanned Surface Vessel Via Reinforcement Learning

Adaptive optimal formation control for unmanned surface vehicles with guaranteed performance using actor‐critic learning architecture

Adaptive Trajectory Tracking Control for Small Unmanned Underwater Vehicles with Prescribed Performance and Dynamic Compensation

Optimal prescribed-time attitude tracking for multiple spacecraft systems based on adaptive critic algorithm

Reinforcement learning-based finite-time tracking control of an unknown unmanned surface vehicle with input constraints

Adaptive Output-Feedback Control with Prescribed Performance for Trajectory Tracking of Underactuated Surface Vessels.

Adaptive NN State Error PCH Trajectory Tracking Control for Unmanned Surface Vessel with Uncertainties and Input Saturation