Abstract:A mixed adaptive dynamic programming (ADP) scheme based on zero-sum game theory is developed to address optimal control problems of autonomous underwater vehicle (AUV) systems subject to disturbances and safe constraints. By combining prior dynamic knowledge and actual sampled data, the proposed approach effectively mitigates the defect caused by the inaccurate dynamic model and significantly improves the training speed of the ADP algorithm. Initially, the dataset is enriched with sufficient reference data collected based on a nominal model without considering modelling bias. Also, the control object interacts with the real environment and continuously gathers adequate sampled data in the dataset. To comprehensively leverage the advantages of model-based and model-free methods during training, an adaptive tuning factor is introduced based on the dataset that possesses model-referenced information and conforms to the distribution of the real-world environment, which balances the influence of model-based control law and data-driven policy gradient on the direction of policy improvement. As a result, the proposed approach accelerates the learning speed compared to data-driven methods, concurrently also enhancing the tracking performance in comparison to model-based control methods. Moreover, the optimal control problem under disturbances is formulated as a zero-sum game, and the actor-critic-disturbance framework is introduced to approximate the optimal control input, cost function, and disturbance policy, respectively. Furthermore, the convergence property of the proposed algorithm based on the value iteration method is analysed. Finally, an example of AUV path following based on the improved line-of-sight guidance is presented to demonstrate the effectiveness of the proposed method.

Prioritized experience replay based reinforcement learning for adaptive tracking control of autonomous underwater vehicle

An Underactuated AUV Tracking Algorithm Based on Backstepping Adaptive Sliding Mode Control

Adaptive Reinforcement Learning Strategy-Based Sliding Mode Control of Uncertain Euler-Lagrange Systems with Prescribed Performance Guarantees: Autonomous Underwater Vehicles-Based Verification

Adversarial Deep Reinforcement Learning Based Robust Depth Tracking Control for Underactuated Autonomous Underwater Vehicle

Sliding Optimal Tracking Control of Autonomous Underwater Vehicles with Adaptive Dynamic Programming

Deep Reinforcement Learning Based Optimal Trajectory Tracking Control of Autonomous Underwater Vehicle

Event-triggered output-feedback adaptive tracking control of autonomous underwater vehicles using reinforcement learning

Adaptive Trajectory Tracking Control with Novel Heading Angle and Velocity Compensation for Autonomous Underwater Vehicles

Trajectory Tracking of Autonomous Underwater Vehicle under Disturbance Based on Time-Delay Adaptive High-Order Sliding Mode Control

Adaptive Backstepping Terminal Sliding Mode Control Method Based on Recurrent Neural Networks for Autonomous Underwater Vehicle

Attention-Based Meta-Reinforcement Learning for Tracking Control of AUV with Time-Varying Dynamics

Trajectory tracking control of vectored thruster autonomous underwater vehicles based on deep reinforcement learning

Learning-based Tracking Control of AUV: Mixed Policy Improvement and Game-Based Disturbance Rejection

Deep reinforcement learning-based controller for dynamic positioning of an unmanned surface vehicle

Adaptive SNDO-STSM Hierarchical Robust Control of Autonomous Underwater Vehicle: Theory and Experimental Validation

Data-driven Trajectory Tracking Control for Autonomous Underwater Vehicle Based on Iterative Extended State Observer.

Adaptive Trajectory Tracking Control for Small Unmanned Underwater Vehicles with Prescribed Performance and Dynamic Compensation

Adaptive Neural Network Dynamic Surface Trajectory Tracking Control for Underactuated Autonomous Underwater Vehicles

Adaptive Neural Network Iterative Sliding Mode Course Tracking Control for Unmanned Surface Vessels

Command filter based adaptive neural trajectory tracking control of an underactuated underwater vehicle in three-dimensional space

3D Trajectory Tracking Control of an Underactuated AUV Based on Adaptive Neural Network Dynamic Surface