Abstract:This paper investigates the optimal control problem for interconnected systems with unknown system dynamics through a two-stage reinforcement learning method. First, to address the impact of interconnection term, the optimal control problem for interconnected systems is transformed into obtaining the solution of game algebraic Riccati equations within the framework of H-infinity control method. Furthermore, existing optimal control approaches for interconnected systems necessitate precise knowledge of system dynamics, which is difficult to obtain accurately or involves high costs. Thus, we introduce a two-stage reinforcement learning method. The admissible control policies are obtained using the homotopy-based iteration method in the first stage. Then, the optimal control policies are obtained through the policy iteration method in the second stage. The two-stage method not only eliminates the requirement for system dynamics and initial admissible control policies but also ensures convergence speed and accuracy, significantly enhancing its practicality. Finally, a two-machine power system example is provided to validate the feasibility of the two-stage method. Note to Practitioners-Interconnected systems, a class of systems composed of multiple local subsystems, find wide applications in various fields such as power systems, transportation networks, and spatially interconnected systems. Particularly, the optimal control problem of interconnected systems has gradually become a focal point of current research. However, the current research on the optimal control problem of interconnected systems is still constrained by the system dynamics and the initial stability of the system. To relax these limitations, this paper introduces a two-stage method. A homotopy-based iteration approach is employed to obtain control policy and interconnection policy that make the system closed-loop stable, thus achieving the optimal solution. Furthermore, the data-driven approach overcomes the limitations imposed by system dynamics. The feasibility of the two-stage method is illustrated by a two-machine power system model.

$$H_\infty $$ Control Using Reinforcement Learning

Reinforcement Learning-Based $\mathcal{h}_{\infty }$ Control of 2-D Markov Jump Roesser Systems with Optimal Disturbance Attenuation

Output Feedback H∞ Control for Linear Discrete-Time Multi-Player Systems with Multi-Source Disturbances Using Off-Policy Q-Learning.

Off-Policy Reinforcement Learning for $ H_\infty $ Control Design

Model-free $H_{\infty}$ control of Itô stochastic system via off-policy reinforcement learning

H∞ Control for Discrete-time Linear Systems by Integrating Off-policy Q-learning and Zero-sum Game

Off-Policy Reinforcement Learning for &Lt;inline-Formula> &Lt;tex-Math Notation="latex">$ H_\infty $ &Lt;/tex-Math></inline-formula> Control Design

Model-Free $h_{2}/h_{\infty}$ Control of Discrete-Time Stochastic Systems: A Reinforcement Learning Method

Robust policy iteration for continuous-time stochastic $H_\infty$ control problem with unknown dynamics

Reinforcement Learning-Based Control for Nonlinear Discrete-Time Systems with Unknown Control Directions and Control Constraints

Robust Tracking Control and Output Regulation

Minimax Q-learning Design for H∞ Control of Linear Discrete-Time Systems

Off-policy reinforcement learning for H∞ control design.

Data-Driven H-infinity Control with a Real-Time and Efficient Reinforcement Learning Algorithm: An Application to Autonomous Mobility-on-Demand Systems

H∞$$ {h}_{\infty } $$ Optimal Output Tracking Control for Markov Jump Systems: A Reinforcement Learning‐based Approach

$H_{\infty}$ Control for Interconnected Systems with Unknown System Dynamics: A Two-Stage Reinforcement Learning Method

Reinforcement Learning for Finite-Horizon H∞ Tracking Control of Unknown Discrete Linear Time-Varying System

A Novel Policy Iteration Algorithm for Nonlinear Continuous-Time H$\infty$ Control Problem

Output-feedback Q-learning for discrete-time linear H-infinity tracking control: A Stackelberg game approach

Data-Driven &Lt;inline-Formula> &Lt;tex-Math Notation="latex">$h_\infty$ &Lt;/tex-Math></inline-formula> Control for Nonlinear Distributed Parameter Systems

Model-free Reinforcement Learning for H_2/H_∞ Control of Stochastic Discrete-time Systems