Abstract:H{\infty} control of nonlinear continuous-time system depends on the solution of the Hamilton-Jacobi-Isaacs (HJI) equation, which has been proved impossible to obtain a closed-form solution due to the nonlinearity of HJI equation. In order to solve HJI equation, many iterative algorithms were proposed, and most of the algorithms were essentially Newton method when the fixed-point equation was constructed in a Banach space. Newton method is a local optimization method, it has small convergence region and needs the initial guess to be sufficiently close to the solution. Whereas damped Newton method enhances the robustness with respect to initial condition and has larger convergence region. In this paper, a novel reinforcement learning method which is named {\alpha}-policy iteration ({\alpha}-PI) is introduced for solving HJI equation. First, by constructing a damped Newton iteration operator equation, a generalized Bellman equation (GBE) is obtained. The GBE is an extension of bellman equation. And then, by iterating on the GBE, an on-policy {\alpha}-PI reinforcement learning method without using knowledge regarding to the system internal dynamics is proposed. Third, based on the on-policy {\alpha}-PI reinforcement learning method, we develop an off-policy {\alpha}-PI reinforcement learning method without requiring any knowledge of the system dynamics. Finally, the neural-network based adaptive critic implementation schemes of on-policy and off-policy {\alpha}-PI algorithms are derived respectively, and the batch least-squares method is used for calculating the weight parameters of neural networks. The effectiveness of the off-policy {\alpha}-PI algorithm is verified through computer simulation.

Policy Iteration Algorithm for Online Design of Robust Control for a Class of Continuous-Time Nonlinear Systems

Robust Hinfinity Controller Design for Time-Varying Uncertain Linear Systems with Time-Varying State and Control Delays

A learning optimal control scheme for robust stabilization of a class of uncertain nonlinear systems

Robust Iterative Learning Control Design Based on Gradient Method

Neural-Network-Based Online HJB Solution for Optimal Robust Guaranteed Cost Control of Continuous-Time Uncertain Nonlinear Systems

Data-based robust optimal control of continuous-time affine nonlinear systems with matched uncertainties

Neural-network-based robust optimal control design for a class of uncertain nonlinear systems via adaptive dynamic programming

A neural-network-based online optimal control approach for nonlinear robust decentralized stabilization

Adaptive optimal control for a class of nonlinear partially uncertain dynamic systems via policy iteration

Robust Adaptive Iterative Learning Control for Discrete‐time Nonlinear Systems with Both Parametric and Nonparametric Uncertainties

Adaptive Optimal Control for a Class of Continuous-Time Affine Nonlinear Systems with Unknown Internal Dynamics

Online Off-Policy Reinforcement Learning for Optimal Control of Unknown Nonlinear Systems Using Neural Networks

Ternary Policy Iteration Algorithm for Nonlinear Robust Control

Neural-model based robust H/sub /spl infin// controllers for discrete-time nonlinear systems: an BMI approach

Online Reinforcement Learning-based Neural Network Controller Design for Affine Nonlinear Discrete-time Systems.

Robust control for a class of nonlinear systems with input constraints based on actor‐critic learning

Robust optimal control for a class of nonlinear dynamic systems using single network adaptive dynamic programming

Robust stabilization of discrete-time uncertain nonlinear systems using neuro-optimal control strategy

Robust Adaptive Iterative Learning Control for Discrete-Time Nonlinear Systems With Time-Iteration-Varying Parameters.

A Novel Policy Iteration Algorithm for Nonlinear Continuous-Time H$\infty$ Control Problem

Adaptive dynamic programming for infinite horizon optimal robust guaranteed cost control of a class of uncertain nonlinear systems