Abstract:The H∞ control method is an effective approach for attenuating the effect of disturbances on practical systems, but it is difficult to obtain the H∞ controller due to the nonlinear Hamilton–Jacobi–Isaacs equation, even for linear systems. This study deals with the design of an H∞ controller for linear discrete-time systems. To solve the related game algebraic Riccati equation (GARE), a novel model-free minimax Q-learning method is developed, on the basis of an offline policy iteration algorithm, which is shown to be Newton’s method for solving the GARE. The proposed minimax Q-learning method, which employs off-policy reinforcement learning, learns the optimal control policies for the controller and the disturbance online, using only the state samples generated by the implemented behavior policies. Different from existing Q-learning methods, a novel gradient-based policy improvement scheme is proposed. We prove that the minimax Q-learning method converges to the saddle solution under initially admissible control policies and an appropriate positive learning rate, provided that certain persistence of excitation (PE) conditions are satisfied. In addition, the PE conditions can be easily met by choosing appropriate behavior policies containing certain excitation noises, without causing any excitation noise bias. In the simulation study, we apply the proposed minimax Q-learning method to design an H∞ load-frequency controller for an electrical power system generator that suffers from load disturbance, and the simulation results indicate that the obtained H∞ load-frequency controller has good disturbance rejection performance.

H∞ Control for Discrete-time Linear Systems by Integrating Off-policy Q-learning and Zero-sum Game

Minimax Q-learning Design for H∞ Control of Linear Discrete-Time Systems

H∞ Tracking Control for Linear Discrete-Time Systems: Model-Free Q-Learning Designs

Output-feedback Q-learning for discrete-time linear H-infinity tracking control: A Stackelberg game approach

Data-Driven H∞ Optimal Output Feedback Control for Linear Discrete-Time Systems Based on Off-Policy Q-Learning

H∞output Feedback Fault-Tolerant Control of Industrial Processes Based on Zero-Sum Games and Off-Policy Q-learning

Off-Policy Reinforcement Learning for $ H_\infty $ Control Design

Output‐feedback Q‐learning for discrete‐time linear <i>H</i><sup>∞</sup> tracking control: A Stackelberg game approach

Off-policy reinforcement learning for H∞ control design.

Optimal H∞ Tracking Control of Nonlinear Systems with Zero-Equilibrium-free Via Novel Adaptive Critic Designs.

Suboptimal H∞ Dynamic Output Feedback Control for Singularly Perturbed Systems

Direct iterative LMI-based approach of mixed H-two / H-infinity robust control for uncertain systems

Learning-Based Nonlinear $H^\infty$ Control via Game-Theoretic Differential Dynamic Programming

Reinforcement Learning for Finite-Horizon H∞ Tracking Control of Unknown Discrete Linear Time-Varying System

H∞ Control with Constrained Input for Completely Unknown Nonlinear Systems Using Data-Driven Reinforcement Learning Method

Neural-network-based Zero-Sum Game for Discrete-Time Nonlinear Systems Via Iterative Adaptive Dynamic Programming Algorithm

Data-Driven &Lt;inline-Formula> &Lt;tex-Math Notation="latex">$h_\infty$ &Lt;/tex-Math></inline-formula> Control for Nonlinear Distributed Parameter Systems

Data-Driven Control for Nonlinear Distributed Parameter Systems

Fuzzy H∞ Control of Discrete-Time Nonlinear Markov Jump Systems via a Novel Hybrid Reinforcement Q-Learning Method

Off-Policy Reinforcement Learning for &Lt;inline-Formula> &Lt;tex-Math Notation="latex">$ H_\infty $ &Lt;/tex-Math></inline-formula> Control Design

Event-driven H ∞ control with critic learning for nonlinear systems