Abstract:A mixed adaptive dynamic programming (ADP) scheme based on zero-sum game theory is developed to address optimal control problems of autonomous underwater vehicle (AUV) systems subject to disturbances and safe constraints. By combining prior dynamic knowledge and actual sampled data, the proposed approach effectively mitigates the defect caused by the inaccurate dynamic model and significantly improves the training speed of the ADP algorithm. Initially, the dataset is enriched with sufficient reference data collected based on a nominal model without considering modelling bias. Also, the control object interacts with the real environment and continuously gathers adequate sampled data in the dataset. To comprehensively leverage the advantages of model-based and model-free methods during training, an adaptive tuning factor is introduced based on the dataset that possesses model-referenced information and conforms to the distribution of the real-world environment, which balances the influence of model-based control law and data-driven policy gradient on the direction of policy improvement. As a result, the proposed approach accelerates the learning speed compared to data-driven methods, concurrently also enhancing the tracking performance in comparison to model-based control methods. Moreover, the optimal control problem under disturbances is formulated as a zero-sum game, and the actor-critic-disturbance framework is introduced to approximate the optimal control input, cost function, and disturbance policy, respectively. Furthermore, the convergence property of the proposed algorithm based on the value iteration method is analysed. Finally, an example of AUV path following based on the improved line-of-sight guidance is presented to demonstrate the effectiveness of the proposed method.

An Improved Adaptive Dynamic Window Algorithm for Target Tracking of Unmanned Surface Vehicles

Improved Line-of-sight Nonlinear Trajectory Tracking Control of Autonomous Underwater Vehicle Exposed to High Variable Speed Ocean Currents

Design on Adaptive Weighted Guidance Law for Underwater Intelligent Vehicle Tracking Target

Dynamic Target Tracking of Small UAVs in Unstructured Environment

Improved A* and Fuzzy Dynamic Window Based Dynamic Trajectory Planning for an UAV

Collision Avoidance and Path Point Tracking Control for Underactuated Unmanned Surface Vehicles with Unknown Model Nonlinearity

A Target Tracking Guidance for Unmanned Surface Vehicles in the Presence of Obstacles

Real-time Obstacle Avoidance for AUV Based on Reinforcement Learning and Dynamic Window Approach

An Improved Target Tracking Algorithm Based on Extended Kalman Filter for UAV

Research on Path Planning Method of Unmanned Boat Based on Improved DWA Algorithm

Terminal Line-of-Sight Angle-Constrained Target Tracking Guidance for Unmanned Surface Vehicles

An adaptive dynamic window approach for UUV obstacle avoidance planning in 3D environments

Event-triggered adaptive dynamic programming for optimal tracking control of unmanned surface vessel with input constraints

Improved Dynamic Window Approach for Unmanned Surface Vehicles' Local Path Planning Considering the Impact of Environmental Factors

Learning-based Tracking Control of AUV: Mixed Policy Improvement and Game-Based Disturbance Rejection

Vector Field-Based Integral LOS Path Following and Target Tracking for Underactuated Unmanned Surface Vehicle

Three-dimensional Underwater Dynamic Target Tracking Based on Adaptive Interactive Multi-model Algorithm

Soft Actor–Critic Based Active Disturbance Rejection Path Following Control for Unmanned Surface Vessel under Wind and Wave Disturbances

An Adaptive Line-of-Sight Guidance Law for Trajectory Tracking of USV with LADRC

Dynamic Optimal Obstacle Avoidance Control of AUV Formation Based on MLoTFWA Algorithm

Sliding Optimal Tracking Control of Autonomous Underwater Vehicles with Adaptive Dynamic Programming