Abstract:H{\infty} control of nonlinear continuous-time system depends on the solution of the Hamilton-Jacobi-Isaacs (HJI) equation, which has been proved impossible to obtain a closed-form solution due to the nonlinearity of HJI equation. In order to solve HJI equation, many iterative algorithms were proposed, and most of the algorithms were essentially Newton method when the fixed-point equation was constructed in a Banach space. Newton method is a local optimization method, it has small convergence region and needs the initial guess to be sufficiently close to the solution. Whereas damped Newton method enhances the robustness with respect to initial condition and has larger convergence region. In this paper, a novel reinforcement learning method which is named {\alpha}-policy iteration ({\alpha}-PI) is introduced for solving HJI equation. First, by constructing a damped Newton iteration operator equation, a generalized Bellman equation (GBE) is obtained. The GBE is an extension of bellman equation. And then, by iterating on the GBE, an on-policy {\alpha}-PI reinforcement learning method without using knowledge regarding to the system internal dynamics is proposed. Third, based on the on-policy {\alpha}-PI reinforcement learning method, we develop an off-policy {\alpha}-PI reinforcement learning method without requiring any knowledge of the system dynamics. Finally, the neural-network based adaptive critic implementation schemes of on-policy and off-policy {\alpha}-PI algorithms are derived respectively, and the batch least-squares method is used for calculating the weight parameters of neural networks. The effectiveness of the off-policy {\alpha}-PI algorithm is verified through computer simulation.

Policy Iteration for Multiplicative Noise Output Feedback Control

Iterative design of suboptimal feedback control for bilinear parabolic PDE systems

A policy iteration algorithm for non-Markovian control problems

Policy Iteration Based Feedback Control

Approximate Midpoint Policy Iteration for Linear Quadratic Control

Policy Gradient Methods for Designing Dynamic Output Feedback Controllers

A Novel Policy Iteration Algorithm for Nonlinear Continuous-Time H$\infty$ Control Problem

Physics-Informed Neural Network Policy Iteration: Algorithms, Convergence, and Verification

Feedback nonlinear discrete-time systems

From Optimization to Control: Quasi Policy Iteration

Multi-Objective Optimal Control with Arbitrary Additive and Multiplicative Noise

Data-Driven H∞ Optimal Output Feedback Control for Linear Discrete-Time Systems Based on Off-Policy Q-Learning

Sparse optimal control of networks with multiplicative noise via policy gradient

Explicit Solution for Constrained Stochastic Linear-Quadratic Control with Multiplicative Noise

Data-driven policy iteration algorithm for continuous-time stochastic linear-quadratic optimal control problems

Reinforcement Learning for Adaptive Optimal Stationary Control of Linear Stochastic Systems

Model-Free $δ$-Policy Iteration Based on Damped Newton Method for Nonlinear Continuous-Time H$\infty$ Tracking Control

Robust Control Design for Linear Systems via Multiplicative Noise

Robust Policy Optimization in Continuous-time Mixed $\mathcal{H}_2/\mathcal{H}_\infty$ Stochastic Control

Continuous-Time Distributed Policy Iteration for Multicontroller Nonlinear Systems