Abstract:H{\infty} control of nonlinear continuous-time system depends on the solution of the Hamilton-Jacobi-Isaacs (HJI) equation, which has been proved impossible to obtain a closed-form solution due to the nonlinearity of HJI equation. In order to solve HJI equation, many iterative algorithms were proposed, and most of the algorithms were essentially Newton method when the fixed-point equation was constructed in a Banach space. Newton method is a local optimization method, it has small convergence region and needs the initial guess to be sufficiently close to the solution. Whereas damped Newton method enhances the robustness with respect to initial condition and has larger convergence region. In this paper, a novel reinforcement learning method which is named {\alpha}-policy iteration ({\alpha}-PI) is introduced for solving HJI equation. First, by constructing a damped Newton iteration operator equation, a generalized Bellman equation (GBE) is obtained. The GBE is an extension of bellman equation. And then, by iterating on the GBE, an on-policy {\alpha}-PI reinforcement learning method without using knowledge regarding to the system internal dynamics is proposed. Third, based on the on-policy {\alpha}-PI reinforcement learning method, we develop an off-policy {\alpha}-PI reinforcement learning method without requiring any knowledge of the system dynamics. Finally, the neural-network based adaptive critic implementation schemes of on-policy and off-policy {\alpha}-PI algorithms are derived respectively, and the batch least-squares method is used for calculating the weight parameters of neural networks. The effectiveness of the off-policy {\alpha}-PI algorithm is verified through computer simulation.

A novel implicit iterative algorithm for solving the generalized Lyapunov equation

A Successive over Relaxation Implicit Iterative Algorithm for Solving Stochastic Linear Systems with Markov Jumps

Iterative Solutions of Coupled Discrete Markovian Jump Lyapunov Equations

Low-rank generalized alternating direction implicit iteration method for solving matrix equations

Wilkinson’s Iterative Refinement of Solution with Automatic Step-Size Control for Linear System of Equations

Adaptive Iterative Learning Control Mechanism for Nonlinear Systems Subject to High-Order Internal Model

A Note on Wilkinson's Iterative Refinement of Solution with Automatic Step-Size Control

Adaptive Iterative Learning Control for Discrete‐time Nonlinear Systems with Multiple Iteration‐varying High‐order Internal Models

An iterative method to solve Lyapunov equations

A low-rank algorithm for solving Lyapunov operator $φ$-functions within the matrix-valued exponential integrators

A general alternating direction implicit iteration method for solving complex symmetric linear systems

Randomized iterative methods for generalized absolute value equations: Solvability and error bounds

The Existence the Solution of Nonlinear Discrete Schemes and Convergence of a Linearized Iterative Method for time-dependent PNP Equations

A Novel Iterative Method for Computing Generalized Inverse

A Novel Policy Iteration Algorithm for Nonlinear Continuous-Time H$\infty$ Control Problem

An Ulm-like algorithm for generalized inverse eigenvalue problems

Relaxed Policy Iteration Algorithm for Nonlinear Zero-Sum Games with Application to H-infinity Control

Extended version with the analysis of dynamic system for iterative refinement of solution

A generalization of the Newton-based matrix splitting iteration method for generalized absolute value equations

An efficient hybrid implicit time integration method with high accuracy for linear and nonlinear dynamics