Abstract:This paper presents a new formulation for model-free robust optimal regulation of continuous-time nonlinear systems. The proposed reinforcement learning based approach, referred to as incremental adaptive dynamic programming (IADP), exploits measured data to allow the design of the approximate optimal incremental control strategy, which stabilizes the controlled system incrementally under model uncertainties, environmental disturbances, and input saturation. By leveraging the time delay estimation (TDE) technique, we first exploit sensory data to reduce the requirement of a complete dynamics, where measured data are adopted to construct an incremental dynamics that reflects the system evolution in an incremental form. Then, the resulting incremental dynamics serves to design the approximate optimal incremental control strategy based on adaptive dynamic programming, which is implemented as a simplified single critic structure to get the approximate solution to the value function of the Hamilton-Jacobi-Bellman equation. Furthermore, for the critic artificial neural network, experience data are used to design an off-policy weight update law with guaranteed weight convergence. Rather importantly, to address the unintentionally introduced TDE error, we incorporate a TDE error bound related term into the cost function, whereby the TDE error is attenuated during the optimization process. The system stability proof and the weight convergence proof are provided. Numerical simulations are conducted to validate the effectiveness and superiority of our proposed IADP, especially regarding the reduced control energy expenditure and the enhanced robustness.

Adaptive Optimal Discrete-Time Output-Feedback Using an Internal Model Principle and Adaptive Dynamic Programming

Adaptive Output Feedback Control for Uncertain Linear Time-Delay Systems

Multi-loop Adaptive Internal Model Control Based on a Dynamic Partial Least Squares Model

Adaptive Dynamic Programming and Adaptive Optimal Output Regulation of Linear Systems

Model-free Adaptive Dynamic Programming for Optimal Control of Discrete-time Affine Nonlinear System

Output Constrained Adaptive Dynamic Programming For Continuous-Time Nonlinear Systems

Improved Dynamic Output Feedback Rmpc for Linear Uncertain Systems with Input Constraints

Adaptive Dynamic Programming and Data-Driven Cooperative Optimal Output Regulation with Adaptive Observers

Robust ADP Design for Continuous-Time Nonlinear Systems with Output Constraints

ADP-based nonlinear optimal output regulation with nonlinear exosystem

Optimal dynamic output feedback control of unknown linear continuous-time systems by adaptive dynamic programming

Model-Free Incremental Adaptive Dynamic Programming Based Approximate Robust Optimal Regulation

Adaptive Optimal Stabilization Regulation of Continuous-Time Systems via Virtual System and Measurement Feedback

Intermittent Feedback Optimal Control of Saturated-Input Nonlinear Systems via Adaptive Dynamic Programming

Data-Driven Event-Triggered Adaptive Dynamic Programming Control for Nonlinear Systems with Input Saturation.

Adaptive Semiglobal Nonlinear Output Regulation: an Extended-State Observer Approach.

A Method for Adaptive Hybrid Output Regulation of Linear Systems

Adaptive Dynamic Programming for Model-Free Global Stabilization of Control Constrained Continuous-Time Systems

Event-Triggered Output Feedback Control for a Class of Nonlinear Systems via Disturbance Observer and Adaptive Dynamic Programming

Near Optimal Neural Network-based Output Feedback Control of Affine Nonlinear Discrete-Time Systems

An Explicit Optimizing Control Approach Based on Output Feedback