Abstract:This paper presents a new formulation for model-free robust optimal regulation of continuous-time nonlinear systems. The proposed reinforcement learning based approach, referred to as incremental adaptive dynamic programming (IADP), exploits measured data to allow the design of the approximate optimal incremental control strategy, which stabilizes the controlled system incrementally under model uncertainties, environmental disturbances, and input saturation. By leveraging the time delay estimation (TDE) technique, we first exploit sensory data to reduce the requirement of a complete dynamics, where measured data are adopted to construct an incremental dynamics that reflects the system evolution in an incremental form. Then, the resulting incremental dynamics serves to design the approximate optimal incremental control strategy based on adaptive dynamic programming, which is implemented as a simplified single critic structure to get the approximate solution to the value function of the Hamilton-Jacobi-Bellman equation. Furthermore, for the critic artificial neural network, experience data are used to design an off-policy weight update law with guaranteed weight convergence. Rather importantly, to address the unintentionally introduced TDE error, we incorporate a TDE error bound related term into the cost function, whereby the TDE error is attenuated during the optimization process. The system stability proof and the weight convergence proof are provided. Numerical simulations are conducted to validate the effectiveness and superiority of our proposed IADP, especially regarding the reduced control energy expenditure and the enhanced robustness.

Data-Driven Direct Adaptive Risk-Sensitive Control of Stochastic Systems

Data-Driven Event-Triggered Adaptive Dynamic Programming Control for Nonlinear Systems with Input Saturation.

Data-Driven Method for Response Control of Nonlinear Random Dynamical Systems

Online Data-Driven Adaptive Control for Unknown Linear Time-Varying Systems

Adaptive Neural Dynamic Surface Control With Prespecified Tracking Accuracy of Uncertain Stochastic Nonstrict-Feedback Systems

Discrete-time adaptive iterative learning control with unknown control directions

Robust ADP Design for Continuous-Time Nonlinear Systems with Output Constraints

Discrete-Time Adaptive Iterative Learning Control for High-Order Nonlinear Systems with Unknown Control Directions

Adaptive Control for Discrete-Time Nonlinear Systems with Non-Sector Nonlinearities and Unknown Input Gain

Adaptive Stochastic Predictive Control from Noisy Data: A Sampling-based Approach

Adaptive Neural Asymptotic Tracking Control for a Class of Stochastic Non-Strict-feedback Switched Systems

Distributionally Robust Stochastic Data-Driven Predictive Control with Optimized Feedback Gain

Twin Deterministic Policy Gradient Adaptive Dynamic Programming for Optimal Control of Affine Nonlinear Discrete-time Systems

Data‐driven adaptive optimal control for discrete‐time periodic systems

Online Adaptive Optimal Control for Continuous-Time Nonlinear Systems with Completely Unknown Dynamics.

Data-Driven Adversarial Online Control for Unknown Linear Systems

Model-Free Incremental Adaptive Dynamic Programming Based Approximate Robust Optimal Regulation

Robust adaptive dynamic programming for optimal nonlinear control design

Accelerated Gradient Approach For Deep Neural Network-Based Adaptive Control of Unknown Nonlinear Systems

Robust Data-Driven Predictive Control for Unknown Linear Time-Invariant Systems

Data-based robust optimal control of continuous-time affine nonlinear systems with matched uncertainties