Abstract:H{\infty} control of nonlinear continuous-time system depends on the solution of the Hamilton-Jacobi-Isaacs (HJI) equation, which has been proved impossible to obtain a closed-form solution due to the nonlinearity of HJI equation. In order to solve HJI equation, many iterative algorithms were proposed, and most of the algorithms were essentially Newton method when the fixed-point equation was constructed in a Banach space. Newton method is a local optimization method, it has small convergence region and needs the initial guess to be sufficiently close to the solution. Whereas damped Newton method enhances the robustness with respect to initial condition and has larger convergence region. In this paper, a novel reinforcement learning method which is named {\alpha}-policy iteration ({\alpha}-PI) is introduced for solving HJI equation. First, by constructing a damped Newton iteration operator equation, a generalized Bellman equation (GBE) is obtained. The GBE is an extension of bellman equation. And then, by iterating on the GBE, an on-policy {\alpha}-PI reinforcement learning method without using knowledge regarding to the system internal dynamics is proposed. Third, based on the on-policy {\alpha}-PI reinforcement learning method, we develop an off-policy {\alpha}-PI reinforcement learning method without requiring any knowledge of the system dynamics. Finally, the neural-network based adaptive critic implementation schemes of on-policy and off-policy {\alpha}-PI algorithms are derived respectively, and the batch least-squares method is used for calculating the weight parameters of neural networks. The effectiveness of the off-policy {\alpha}-PI algorithm is verified through computer simulation.

Bias-policy iteration based adaptive dynamic programming for unknown continuous-time linear systems

Bias-policy iteration based optimal control for unknown continuous-time linear periodic systems

Model-free Adaptive Dynamic Programming for Optimal Control of Discrete-time Affine Nonlinear System

A Learning-Based Optimal Tracking Controller for Continuous Linear Systems with Unknown Dynamics: Theory and Case Study

Modified λ-Policy Iteration Based Adaptive Dynamic Programming for Unknown Discrete-Time Linear Systems

Modified $\lambda$-Policy Iteration Based Adaptive Dynamic Programming for Unknown Discrete-Time Linear Systems

Modified general policy iteration based adaptive dynamic programming for unknown discrete‐time linear systems

Policy-Iteration-Based Finite-Horizon Approximate Dynamic Programming for Continuous-Time Nonlinear Optimal Control

Discrete-Time Adaptive Iterative Learning Control for High-Order Nonlinear Systems with Unknown Control Directions

Optimal dynamic output feedback control of unknown linear continuous-time systems by adaptive dynamic programming

Policy Iteration Based Feedback Control

Twin Deterministic Policy Gradient Adaptive Dynamic Programming for Optimal Control of Affine Nonlinear Discrete-time Systems

Data-based robust optimal control of continuous-time affine nonlinear systems with matched uncertainties

Learning-based adaptive optimal control of linear time-delay systems: A value iteration approach

Adaptive Iterative Learning Control for Discrete‐time Nonlinear Systems with Multiple Iteration‐varying High‐order Internal Models

Solving optimal predictor-feedback control using approximate dynamic programming

H ∞ optimal control of unknown linear systems by adaptive dynamic programming with applications to time‐delay systems

On policy iteration‐based discounted optimal control

A New Continuous-Time Policy Iteration for Time-Varying Nonlinear Systems

Data-driven policy iteration algorithm for continuous-time stochastic linear-quadratic optimal control problems

A Novel Policy Iteration Algorithm for Nonlinear Continuous-Time H$\infty$ Control Problem