Abstract:H{\infty} control of nonlinear continuous-time system depends on the solution of the Hamilton-Jacobi-Isaacs (HJI) equation, which has been proved impossible to obtain a closed-form solution due to the nonlinearity of HJI equation. In order to solve HJI equation, many iterative algorithms were proposed, and most of the algorithms were essentially Newton method when the fixed-point equation was constructed in a Banach space. Newton method is a local optimization method, it has small convergence region and needs the initial guess to be sufficiently close to the solution. Whereas damped Newton method enhances the robustness with respect to initial condition and has larger convergence region. In this paper, a novel reinforcement learning method which is named {\alpha}-policy iteration ({\alpha}-PI) is introduced for solving HJI equation. First, by constructing a damped Newton iteration operator equation, a generalized Bellman equation (GBE) is obtained. The GBE is an extension of bellman equation. And then, by iterating on the GBE, an on-policy {\alpha}-PI reinforcement learning method without using knowledge regarding to the system internal dynamics is proposed. Third, based on the on-policy {\alpha}-PI reinforcement learning method, we develop an off-policy {\alpha}-PI reinforcement learning method without requiring any knowledge of the system dynamics. Finally, the neural-network based adaptive critic implementation schemes of on-policy and off-policy {\alpha}-PI algorithms are derived respectively, and the batch least-squares method is used for calculating the weight parameters of neural networks. The effectiveness of the off-policy {\alpha}-PI algorithm is verified through computer simulation.

A New Continuous-Time Policy Iteration for Time-Varying Nonlinear Systems

Continuous-Time Policy Optimization.

Local Policy Iteration Adaptive Dynamic Programming for Discrete-Time Nonlinear Systems

Policy-Iteration-Based Finite-Horizon Approximate Dynamic Programming for Continuous-Time Nonlinear Optimal Control

Continuous-Time Distributed Policy Iteration for Multicontroller Nonlinear Systems

A Novel Policy Iteration Algorithm for Nonlinear Continuous-Time H$\infty$ Control Problem

Adaptive Optimal Control for a Class of Nonlinear Systems: the Online Policy Iteration Approach.

Constrained Event-Driven Policy Iteration Design for Nonlinear Discrete Time Systems

Policy Iteration Algorithm for Constrained Cost Optimal Control of Discrete-Time Nonlinear System

Adaptive optimal control for a class of nonlinear partially uncertain dynamic systems via policy iteration

Relaxed Actor-Critic with Convergence Guarantees for Continuous-Time Optimal Control of Nonlinear Systems.

Online Policy Iterative-Based H∞ Optimization Algorithm for a Class of Nonlinear Systems

Nearly Optimal Tracking Control For Continuous Time Nonlinear Systems Using A Policy Iteration Based Hjb Approach

Nonlinear Neuro-Optimal Tracking Control via Stable Iterative Q-Learning Algorithm

Discrete-Time Nonlinear Generalized Policy Iteration for Optimal Control Using Neural Networks

Modified λ-Policy Iteration Based Adaptive Dynamic Programming for Unknown Discrete-Time Linear Systems

Ternary Policy Iteration Algorithm for Nonlinear Robust Control

Infinite Horizon Self-Learning Optimal Control of Nonaffine Discrete-Time Nonlinear Systems

Neuro-Control for Continuous-Time Stochastic Nonlinear Systems Via Online Policy Iteration Algorithm

Modified $\lambda$-Policy Iteration Based Adaptive Dynamic Programming for Unknown Discrete-Time Linear Systems

Two‐loop reinforcement learning algorithm for finite‐horizon optimal control of continuous‐time affine nonlinear systems