Abstract:This paper presents a new formulation for model-free robust optimal regulation of continuous-time nonlinear systems. The proposed reinforcement learning based approach, referred to as incremental adaptive dynamic programming (IADP), exploits measured data to allow the design of the approximate optimal incremental control strategy, which stabilizes the controlled system incrementally under model uncertainties, environmental disturbances, and input saturation. By leveraging the time delay estimation (TDE) technique, we first exploit sensory data to reduce the requirement of a complete dynamics, where measured data are adopted to construct an incremental dynamics that reflects the system evolution in an incremental form. Then, the resulting incremental dynamics serves to design the approximate optimal incremental control strategy based on adaptive dynamic programming, which is implemented as a simplified single critic structure to get the approximate solution to the value function of the Hamilton-Jacobi-Bellman equation. Furthermore, for the critic artificial neural network, experience data are used to design an off-policy weight update law with guaranteed weight convergence. Rather importantly, to address the unintentionally introduced TDE error, we incorporate a TDE error bound related term into the cost function, whereby the TDE error is attenuated during the optimization process. The system stability proof and the weight convergence proof are provided. Numerical simulations are conducted to validate the effectiveness and superiority of our proposed IADP, especially regarding the reduced control energy expenditure and the enhanced robustness.

Adaptive Dynamic Programming and Adaptive Optimal Output Regulation of Linear Systems

Decentralized Adaptive Output Regulation for a Class of Large-Scale Nonlinear Systems.

Optimal Output Regulation for General Linear Systems Via Adaptive Dynamic Programming

Adaptive Dynamic Programming-Based Robust Output Regulation of Discrete-Time Linear Systems via Output Feedback

Optimal Output Regulation for Unknown Continuous-Time Linear Systems by Internal Model and Adaptive Dynamic Programming

Value iteration and adaptive optimal output regulation with assured convergence rate

A Learning-Based Optimal Tracking Controller for Continuous Linear Systems with Unknown Dynamics: Theory and Case Study

Adaptive optimal output regulation of linear discrete-time systems based on event-triggered output-feedback

Adaptive Optimal Output Tracking Control of Completely Unknown Linear Two-Time-Scale Systems

ADP-based nonlinear optimal output regulation with nonlinear exosystem

Learning-based Event-triggered Adaptive Optimal Output Regulation of Linear Discrete-time Systems

Optimal Output Tracking Control of Linear Discrete-Time Systems with Unknown Dynamics by Adaptive Dynamic Programming and Output Feedback

Adaptive Output-Feedback Optimal Control for Continuous-Time Linear Systems Based on Adaptive Dynamic Programming Approach

Output Constrained Adaptive Dynamic Programming For Continuous-Time Nonlinear Systems

Adaptive Optimal Output Regulation of Interconnected Singularly Perturbed Systems with Application to Power Systems

Adaptive Dynamic Programming and Decentralized Optimal Output Regulation of Two-Time-Scale Interconnected Systems

Optimal Output Regulation of Linear Discrete-Time Systems with Unknown Dynamics Using Reinforcement Learning

New Results in Cooperative Adaptive Optimal Output Regulation

Optimal dynamic output feedback control of unknown linear continuous-time systems by adaptive dynamic programming

A Method for Adaptive Hybrid Output Regulation of Linear Systems

Model-Free Incremental Adaptive Dynamic Programming Based Approximate Robust Optimal Regulation