Abstract:This paper deals with the robust control issues for a class of uncertain nonlinear systems with completely unknown dynamics via a data-driven reinforcement learning method. Firstly, we formulate the optimal regulation control problem for the nominal system, and then, the robust controller for the original uncertain system is designed by adding a constant feedback gain to the optimal controller of the nominal system. Then, this scheme is extended to the optimal tracking control by means of augmented system and discount factor. It is also demonstrated that the proposed robust controller can achieve optimality with a new defined performance index function when there is no control perturbation. It is well known that the nonlinear optimal control problem relies on the solution of Hamilton–Jacobi–Bellman (HJB) equation, which is a nonlinear partial differential equation and impossible to be solved analytically. In order to overcome this difficulty, we introduce a model-based iterative learning algorithm to successively approximate the solution of HJB equation and provide its convergence proof. Subsequently, based on the structure of the model-based approach, a data-driven reinforcement learning method is derived, which only requires the sampling data from real system with different control inputs rather than the accurate mathematical system models. Neural networks (NNs) are utilized to implement this model-free method to approximate the optimal solutions and the least-square approach is employed to minimize the NN approximation residual errors. Finally, two numerical simulation examples are given to illustrate the effectiveness of our proposed method.

Data-based reinforcement learning approximate optimal control for an uncertain nonlinear system with control effectiveness faults

A Learning-Based Optimal Tracking Controller for Continuous Linear Systems with Unknown Dynamics: Theory and Case Study

Data-Driven Optimal Fault-Tolerant Control for Unknown Linear Systems

Data-driven iterative adaptive dynamic programming algorithm for approximate optimal control of unknown nonlinear systems

Model-based reinforcement learning for infinite-horizon approximate optimal tracking

Data-Based Approximate Optimal Control for Unknown Nonaffine Systems via Dynamic Feedback

Robust control scheme for a class of uncertain nonlinear systems with completely unknown dynamics using data-driven reinforcement learning method.

Robust Adaptive Iterative Learning Control for Discrete‐time Nonlinear Systems with Both Parametric and Nonparametric Uncertainties

Data-based Robust Adaptive Control for a Class of Unknown Nonlinear Constrained-Input Systems Via Integral Reinforcement Learning

Data-Driven Robust Approximate Optimal Tracking Control for Unknown General Nonlinear Systems Using Adaptive Dynamic Programming Method

Adaptive Fault-Tolerant Control for Affine Non-Linear Systems Based on Approximate Dynamic Programming

Data-Driven Near-Optimal Control of Nonlinear Systems Over Finite Horizon

Data-Based Self-Learning Optimal Control For Continuous-Time Unknown Nonlinear Systems With Disturbance

Data-based robust control for unknown nonlinear systems

Model-Free Incremental Adaptive Dynamic Programming Based Approximate Robust Optimal Regulation

Adaptive neural fault‐tolerant optimal control for nonlinear uncertain systems with dynamic state constraints

Dynamic Programming-based Approximate Optimal Control for Model-Based Reinforcement Learning

Robust Near-optimal Control for Constrained Nonlinear System via Integral Reinforcement Learning

Recursive terminal sliding mode approximate optimal control strategy based on reconstructed nonlinear systems

Dynamic Compensator-Based Near-Optimal Control for Unknown Nonaffine Systems Via Integral Reinforcement Learning

Neural-network-based Control Scheme for a Class of Nonlinear Systems with Actuator Faults Via Data-Driven Reinforcement Learning Method