Abstract:The H∞ control method is an effective approach for attenuating the effect of disturbances on practical systems, but it is difficult to obtain the H∞ controller due to the nonlinear Hamilton–Jacobi–Isaacs equation, even for linear systems. This study deals with the design of an H∞ controller for linear discrete-time systems. To solve the related game algebraic Riccati equation (GARE), a novel model-free minimax Q-learning method is developed, on the basis of an offline policy iteration algorithm, which is shown to be Newton’s method for solving the GARE. The proposed minimax Q-learning method, which employs off-policy reinforcement learning, learns the optimal control policies for the controller and the disturbance online, using only the state samples generated by the implemented behavior policies. Different from existing Q-learning methods, a novel gradient-based policy improvement scheme is proposed. We prove that the minimax Q-learning method converges to the saddle solution under initially admissible control policies and an appropriate positive learning rate, provided that certain persistence of excitation (PE) conditions are satisfied. In addition, the PE conditions can be easily met by choosing appropriate behavior policies containing certain excitation noises, without causing any excitation noise bias. In the simulation study, we apply the proposed minimax Q-learning method to design an H∞ load-frequency controller for an electrical power system generator that suffers from load disturbance, and the simulation results indicate that the obtained H∞ load-frequency controller has good disturbance rejection performance.

Output Feedback H∞ Control for Linear Discrete-Time Multi-Player Systems with Multi-Source Disturbances Using Off-Policy Q-Learning.

$$H_\infty $$ Control Using Reinforcement Learning

Data-Driven H∞ Optimal Output Feedback Control for Linear Discrete-Time Systems Based on Off-Policy Q-Learning

Output Feedback H∞ Control for Discrete Time Singularly Perturbed Systems with Markov Lossy Network: the Round-Robin-like Protocol Case

H∞ Control for Discrete-time Linear Systems by Integrating Off-policy Q-learning and Zero-sum Game

H∞$$ {h}_{\infty } $$ Optimal Output Tracking Control for Markov Jump Systems: A Reinforcement Learning‐based Approach

Output-feedback Q-learning for discrete-time linear H-infinity tracking control: A Stackelberg game approach

Data-Efficient Off-Policy Learning for Distributed Optimal Tracking Control of HMAS with Unidentified Exosystem Dynamics.

Robust Optimal Tracking Control for Multiplayer Systems by Off‐policy Q‐learning Approach

Discrete-Time Multi-Player Games Based on Off-Policy Q-Learning

Robust Optimal Tracking Control for Linear Systems via Adaptive Dynamic Programming method

Minimax Q-learning Design for H∞ Control of Linear Discrete-Time Systems

Reinforcement Learning-Based Control for Nonlinear Discrete-Time Systems with Unknown Control Directions and Control Constraints

Robust Tracking Control and Output Regulation

Output‐feedback Q‐learning for discrete‐time linear <i>H</i><sup>∞</sup> tracking control: A Stackelberg game approach

Robust Control Design for Multi-Player Nonlinear Systems with Input Disturbances Via Adaptive Dynamic Programming

Multi-H∞ Controls for Unknown Input-Interference Nonlinear System with Reinforcement Learning.

Output Feedback Adaptive Dynamic Programming for Linear H∞ Tracking Control

Model-Free Linear Discrete-Time System H∞ Control Using Input-Output Data

Data-Driven Optimal Control for Multi-Player Non-Zero-Sum Games with Unknown Dynamics

Optimal Tracking Control for Non-Zero-sum Games of Linear Discrete-Time Systems Via Off-Policy Reinforcement Learning