Abstract:This paper deals with the robust control issues for a class of uncertain nonlinear systems with completely unknown dynamics via a data-driven reinforcement learning method. Firstly, we formulate the optimal regulation control problem for the nominal system, and then, the robust controller for the original uncertain system is designed by adding a constant feedback gain to the optimal controller of the nominal system. Then, this scheme is extended to the optimal tracking control by means of augmented system and discount factor. It is also demonstrated that the proposed robust controller can achieve optimality with a new defined performance index function when there is no control perturbation. It is well known that the nonlinear optimal control problem relies on the solution of Hamilton–Jacobi–Bellman (HJB) equation, which is a nonlinear partial differential equation and impossible to be solved analytically. In order to overcome this difficulty, we introduce a model-based iterative learning algorithm to successively approximate the solution of HJB equation and provide its convergence proof. Subsequently, based on the structure of the model-based approach, a data-driven reinforcement learning method is derived, which only requires the sampling data from real system with different control inputs rather than the accurate mathematical system models. Neural networks (NNs) are utilized to implement this model-free method to approximate the optimal solutions and the least-square approach is employed to minimize the NN approximation residual errors. Finally, two numerical simulation examples are given to illustrate the effectiveness of our proposed method.

Model-Based Cross-Scale Reinforcement Learning Optimal Control

Optimal Control for Constrained Discrete-Time Nonlinear Systems Based on Safe Reinforcement Learning.

An Adaptive Cross Approximation-based Method for Robust Nonlinear Feedback Control Problems

Reinforcement Learning Based Optimal Control of Linear Singularly Perturbed Systems

A hybrid model-based optimal control method for nonlinear systems using simultaneous dynamic optimization strategies

Online Reinforcement Learning-based Neural Network Controller Design for Affine Nonlinear Discrete-time Systems.

Data-Driven Near-Optimal Control of Nonlinear Systems Over Finite Horizon

Optimal control for continuous-time Markov jump singularly perturbed systems : A hybrid reinforcement learning scheme

Reinforcement Learning-Based Control for Nonlinear Discrete-Time Systems with Unknown Control Directions and Control Constraints

Suboptimal Control of a Class of Nonlinear Singularly Perturbed Systems

A Model-free Framework for Nonlinear Optimal Control Based on Solving HJB Equation

Hybrid Reinforcement Learning for Optimal Control of Non-Linear Switching System

Reinforcement learning‐based composite suboptimal control for Markov jump singularly perturbed systems with unknown dynamics

A Fuzzy-Model-Based Approach to Optimal Control for Nonlinear Markov Jump Singularly Perturbed Systems: A Novel Integral Reinforcement Learning Scheme

Suboptimal Reduced Control of Unknown Nonlinear Singularly Perturbed Systems Via Reinforcement Learning

New Methods for Optimal Operational Control of Industrial Processes Using Reinforcement Learning on Two Time Scales

Reinforcement Learning for Adaptive Optimal Control of Unknown Continuous-Time Nonlinear Systems with Input Constraints.

Learning-based optimal control of linear time-varying systems over large time intervals

Decentralized Optimization for Fuzzy Singularly Perturbed Interconnected Systems: A Hybrid Reinforcement Learning Method

Reinforcement learning for optimal tracking of large-scale systems with multitime scales

Robust control scheme for a class of uncertain nonlinear systems with completely unknown dynamics using data-driven reinforcement learning method.