Abstract:This paper deals with the robust control issues for a class of uncertain nonlinear systems with completely unknown dynamics via a data-driven reinforcement learning method. Firstly, we formulate the optimal regulation control problem for the nominal system, and then, the robust controller for the original uncertain system is designed by adding a constant feedback gain to the optimal controller of the nominal system. Then, this scheme is extended to the optimal tracking control by means of augmented system and discount factor. It is also demonstrated that the proposed robust controller can achieve optimality with a new defined performance index function when there is no control perturbation. It is well known that the nonlinear optimal control problem relies on the solution of Hamilton–Jacobi–Bellman (HJB) equation, which is a nonlinear partial differential equation and impossible to be solved analytically. In order to overcome this difficulty, we introduce a model-based iterative learning algorithm to successively approximate the solution of HJB equation and provide its convergence proof. Subsequently, based on the structure of the model-based approach, a data-driven reinforcement learning method is derived, which only requires the sampling data from real system with different control inputs rather than the accurate mathematical system models. Neural networks (NNs) are utilized to implement this model-free method to approximate the optimal solutions and the least-square approach is employed to minimize the NN approximation residual errors. Finally, two numerical simulation examples are given to illustrate the effectiveness of our proposed method.

Improved Off‐policy Reinforcement Learning Algorithm for Robust Control of Unmodeled Nonlinear System with Asymmetric State Constraints

A Dual Adaptive Robust Control for Nonlinear Systems with Parameter and State Estimation

Adaptive Finite-Time Optimised Impedance Control for Robotic Manipulators with State Constraints

Reinforcement Learning Controller Design for Affine Nonlinear Discrete-Time Systems Using Online Approximators

Direct Optimization Based Compensation Adaptive Robust Control of Nonlinear Systems with State and Input Constraints

Optimal Robust Control of Nonlinear Uncertain System Via Off-Policy Integral Reinforcement Learning

Data-Driven Robust Control of Discrete-Time Uncertain Linear Systems Via Off-Policy Reinforcement Learning.

Online Reinforcement Learning-based Neural Network Controller Design for Affine Nonlinear Discrete-time Systems.

Off-Policy Risk-Sensitive Reinforcement Learning-Based Constrained Robust Optimal Control

Data-based Robust Adaptive Control for a Class of Unknown Nonlinear Constrained-Input Systems Via Integral Reinforcement Learning

Off‐policy reinforcement learning algorithm for robust optimal control of uncertain nonlinear systems

Data-based robust control for unknown nonlinear systems

Reinforcement Learning for Robust Adaptive Control of Partially Unknown Nonlinear Systems Subject to Unmatched Uncertainties

Robust control scheme for a class of uncertain nonlinear systems with completely unknown dynamics using data-driven reinforcement learning method.

Safety-Aware Optimal Control of Nonlinear Systems Using Off-Policy Reinforcement Learning*

Robust Control for Affine Nonlinear Systems under the Reinforcement Learning Framework

Reinforcement Learning for Robust Neuro-Control of Constrained Nonlinear Systems

An Off‐policy Iteration Algorithm for Robust Stabilization of Constrained‐input Uncertain Nonlinear Systems

Reduced-Order Multiple Models Robust Direct Adaptive Control of Non-Minimum Phase System

Model-free Nonlinear Robust Control Design Via Online Critic Learning.

Reinforcement learning based adaptive optimal control for constrained nonlinear system via a novel state-dependent transformation