Abstract:This paper presents a new formulation for model-free robust optimal regulation of continuous-time nonlinear systems. The proposed reinforcement learning based approach, referred to as incremental adaptive dynamic programming (IADP), exploits measured data to allow the design of the approximate optimal incremental control strategy, which stabilizes the controlled system incrementally under model uncertainties, environmental disturbances, and input saturation. By leveraging the time delay estimation (TDE) technique, we first exploit sensory data to reduce the requirement of a complete dynamics, where measured data are adopted to construct an incremental dynamics that reflects the system evolution in an incremental form. Then, the resulting incremental dynamics serves to design the approximate optimal incremental control strategy based on adaptive dynamic programming, which is implemented as a simplified single critic structure to get the approximate solution to the value function of the Hamilton-Jacobi-Bellman equation. Furthermore, for the critic artificial neural network, experience data are used to design an off-policy weight update law with guaranteed weight convergence. Rather importantly, to address the unintentionally introduced TDE error, we incorporate a TDE error bound related term into the cost function, whereby the TDE error is attenuated during the optimization process. The system stability proof and the weight convergence proof are provided. Numerical simulations are conducted to validate the effectiveness and superiority of our proposed IADP, especially regarding the reduced control energy expenditure and the enhanced robustness.

Continuous-Time Robust Dynamic Programming

Robust ADP Design for Continuous-Time Nonlinear Systems with Output Constraints

Robust adaptive dynamic programming for optimal nonlinear control design

Approximate dynamic programming for continuous state and control problems

Robust Optimal Control For Time-Delay Systems With Dynamic Uncertainties Via Adp

Adaptive dynamic programming for infinite horizon optimal robust guaranteed cost control of a class of uncertain nonlinear systems

Model-Free Incremental Adaptive Dynamic Programming Based Approximate Robust Optimal Regulation

Time-Constrained Robust MDPs

Robust Dynamic Programming for Temporal Logic Control of Stochastic Systems

Adaptive Dynamic Programming for Control: A Survey and Recent Advances

Dynamic Programming for General Linear Quadratic Optimal Stochastic Control with Random Coefficients

Event-Based Robust Control for Uncertain Nonlinear Systems Using Adaptive Dynamic Programming

Output Constrained Adaptive Dynamic Programming For Continuous-Time Nonlinear Systems

Neural Dynamic Programming for Event-Based Nonlinear Adaptive Robust Stabilization.

Robust Optimal Parallel Tracking Control Based on Adaptive Dynamic Programming

A continuous time formulation of stochastic dual control to avoid the curse of dimensionality

Event-Triggered Adaptive Dynamic Programming for Continuous-Time Systems with Control Constraints

Continuous-Time Distributed Dynamic Programming for Networked Multi-Agent Markov Decision Processes

Adaptive Dynamic Programming for a Class of Complex-Valued Nonlinear Systems

Event-Triggered Adaptive Dynamic Programming for Unmatched Uncertain Nonlinear Continuous-Time Systems

Robust Distributed Model Predictive Control of Constrained Continuous-Time Nonlinear Systems: A Robustness Constraint Approach.