Abstract:H{\infty} control of nonlinear continuous-time system depends on the solution of the Hamilton-Jacobi-Isaacs (HJI) equation, which has been proved impossible to obtain a closed-form solution due to the nonlinearity of HJI equation. In order to solve HJI equation, many iterative algorithms were proposed, and most of the algorithms were essentially Newton method when the fixed-point equation was constructed in a Banach space. Newton method is a local optimization method, it has small convergence region and needs the initial guess to be sufficiently close to the solution. Whereas damped Newton method enhances the robustness with respect to initial condition and has larger convergence region. In this paper, a novel reinforcement learning method which is named {\alpha}-policy iteration ({\alpha}-PI) is introduced for solving HJI equation. First, by constructing a damped Newton iteration operator equation, a generalized Bellman equation (GBE) is obtained. The GBE is an extension of bellman equation. And then, by iterating on the GBE, an on-policy {\alpha}-PI reinforcement learning method without using knowledge regarding to the system internal dynamics is proposed. Third, based on the on-policy {\alpha}-PI reinforcement learning method, we develop an off-policy {\alpha}-PI reinforcement learning method without requiring any knowledge of the system dynamics. Finally, the neural-network based adaptive critic implementation schemes of on-policy and off-policy {\alpha}-PI algorithms are derived respectively, and the batch least-squares method is used for calculating the weight parameters of neural networks. The effectiveness of the off-policy {\alpha}-PI algorithm is verified through computer simulation.

Online policy iteration algorithm for semi-Markov switching state-space control processes

A nonlinear predictive control algorithm based on fuzzy online modeling and discrete optimization

Online Adaptive Optimization Algorithm for Semi-Markov Control Processes

Optimization Of Semi-Markov Switching State-Space Control Processes For Network Communication Systems

Optimization Algorithms for Semi-Markov Control Processes with Average Criteria

Model-free Adaptive Dynamic Programming for Optimal Control of Discrete-time Affine Nonlinear System

Approximate Policy Iteration for Robust Stochastic Control of Multi-agent Markov Decision Processes

Adaptive Optimization of Time-out Policy for Dynamic Power Management Based on SMCP

On Solving Optimal Policies for Finite-Stage Event-Based Optimization

A policy iteration algorithm for non-Markovian control problems

Policy Iteration Based Feedback Control

From Optimization to Control: Quasi Policy Iteration

Online Policy Optimization in Unknown Nonlinear Systems

Hierarchical Policy Iteration for Large-Scale Pomdp Systems

Online Planning for Decentralized Stochastic Control with Partial History Sharing

Online Adaptive Optimal Control Algorithm Based on Synchronous Integral Reinforcement Learning With Explorations

Optimal Stationary Policies for Semi-Markov Control Processes with Discounted-Cost Criteria

A Novel Policy Iteration Algorithm for Nonlinear Continuous-Time H$\infty$ Control Problem

Simulation Optimization Algorithm for SMDPs with Parameterized Randomized Stationary Policies

Optimal control for continuous-time Markov jump singularly perturbed systems : A hybrid reinforcement learning scheme

Online Markov decision processes with policy iteration