Abstract:A mixed adaptive dynamic programming (ADP) scheme based on zero-sum game theory is developed to address optimal control problems of autonomous underwater vehicle (AUV) systems subject to disturbances and safe constraints. By combining prior dynamic knowledge and actual sampled data, the proposed approach effectively mitigates the defect caused by the inaccurate dynamic model and significantly improves the training speed of the ADP algorithm. Initially, the dataset is enriched with sufficient reference data collected based on a nominal model without considering modelling bias. Also, the control object interacts with the real environment and continuously gathers adequate sampled data in the dataset. To comprehensively leverage the advantages of model-based and model-free methods during training, an adaptive tuning factor is introduced based on the dataset that possesses model-referenced information and conforms to the distribution of the real-world environment, which balances the influence of model-based control law and data-driven policy gradient on the direction of policy improvement. As a result, the proposed approach accelerates the learning speed compared to data-driven methods, concurrently also enhancing the tracking performance in comparison to model-based control methods. Moreover, the optimal control problem under disturbances is formulated as a zero-sum game, and the actor-critic-disturbance framework is introduced to approximate the optimal control input, cost function, and disturbance policy, respectively. Furthermore, the convergence property of the proposed algorithm based on the value iteration method is analysed. Finally, an example of AUV path following based on the improved line-of-sight guidance is presented to demonstrate the effectiveness of the proposed method.

Sliding Optimal Tracking Control of Autonomous Underwater Vehicles with Adaptive Dynamic Programming

A double-loop control framework for AUV trajectory tracking under model parameters uncertainties and time-varying currents

An Underactuated AUV Tracking Algorithm Based on Backstepping Adaptive Sliding Mode Control

Improved Line-of-sight Nonlinear Trajectory Tracking Control of Autonomous Underwater Vehicle Exposed to High Variable Speed Ocean Currents

Adaptive Reinforcement Learning Strategy-Based Sliding Mode Control of Uncertain Euler-Lagrange Systems with Prescribed Performance Guarantees: Autonomous Underwater Vehicles-Based Verification

Learning-based Tracking Control of AUV: Mixed Policy Improvement and Game-Based Disturbance Rejection

Backstepping Sliding Mode Control of Dynamic Positioning of an Autonomous Underwater Vehicle(AUV)

Adaptive Asymptotic Tracking Control for Underactuated Autonomous Underwater Vehicles with State Constraints

Dual-Loop Integral Sliding Mode Control-Based Path Tracking with Reaction Torque for Autonomous Underwater Vehicle

Adaptive Predefined-Time Optimal Tracking Control for Underactuated Autonomous Underwater Vehicles.

Dynamical Sliding Mode Control for the Trajectory Tracking of Underactuated Unmanned Underwater Vehicles

Robust Adaptive Sliding Mode Control of Underactuated Autonomous Underwater Vehicles with Uncertain Dynamics

Three-Dimensional Optimal Trajectory Tracking Control of Underactuated Autonomous Underwater Vehicles Using Double Closed-Loop Control

Adaptive Sliding Mode Control for AUV based on Backstepping and Neural Networks

Dual Closed-Loop Sliding Mode Predictive Tracking Control of Unmanned Underwater Vehicles with Unknown Disturbances

Robust Adaptive Optimal Trajectory Tracking Control for Underactuated AUVs with Position and Velocity Constraints in Three‐dimensional Space

Adaptive sliding-mode trajectory tracking control of underactuated unmanned surface vessels

Data-driven Trajectory Tracking Control for Autonomous Underwater Vehicle Based on Iterative Extended State Observer.

Trajectory Tracking Control of an Autonomous Underwater Vehicle Using Lyapunov-Based Model Predictive Control

Three-dimensional optimal trajectory tracking control of underactuated AUVs with uncertain dynamics and input saturation

Trajectory tracking sliding mode control of underactuated AUVs