Abstract:This paper investigates the optimal control problem for interconnected systems with unknown system dynamics through a two-stage reinforcement learning method. First, to address the impact of interconnection term, the optimal control problem for interconnected systems is transformed into obtaining the solution of game algebraic Riccati equations within the framework of H-infinity control method. Furthermore, existing optimal control approaches for interconnected systems necessitate precise knowledge of system dynamics, which is difficult to obtain accurately or involves high costs. Thus, we introduce a two-stage reinforcement learning method. The admissible control policies are obtained using the homotopy-based iteration method in the first stage. Then, the optimal control policies are obtained through the policy iteration method in the second stage. The two-stage method not only eliminates the requirement for system dynamics and initial admissible control policies but also ensures convergence speed and accuracy, significantly enhancing its practicality. Finally, a two-machine power system example is provided to validate the feasibility of the two-stage method. Note to Practitioners-Interconnected systems, a class of systems composed of multiple local subsystems, find wide applications in various fields such as power systems, transportation networks, and spatially interconnected systems. Particularly, the optimal control problem of interconnected systems has gradually become a focal point of current research. However, the current research on the optimal control problem of interconnected systems is still constrained by the system dynamics and the initial stability of the system. To relax these limitations, this paper introduces a two-stage method. A homotopy-based iteration approach is employed to obtain control policy and interconnection policy that make the system closed-loop stable, thus achieving the optimal solution. Furthermore, the data-driven approach overcomes the limitations imposed by system dynamics. The feasibility of the two-stage method is illustrated by a two-machine power system model.

Data-Driven &Lt;inline-Formula> &Lt;tex-Math Notation="latex">$h_\infty$ &Lt;/tex-Math></inline-formula> Control for Nonlinear Distributed Parameter Systems

Data-Driven Control for Nonlinear Distributed Parameter Systems

H∞ Control with Constrained Input for Completely Unknown Nonlinear Systems Using Data-Driven Reinforcement Learning Method

Data-Driven Method for Response Control of Nonlinear Random Dynamical Systems

Off-Policy Reinforcement Learning for &Lt;inline-Formula> &Lt;tex-Math Notation="latex">$ H_\infty $ &Lt;/tex-Math></inline-formula> Control Design

Off-Policy Reinforcement Learning for $ H_\infty $ Control Design

Nonlinear H∞ Control Based on Successive Gaussian Process Regression

Off-policy reinforcement learning for H∞ control design.

A Novel Policy Iteration Algorithm for Nonlinear Continuous-Time H$\infty$ Control Problem

Online Adaptive Policy Learning Algorithm for H-Infinity State Feedback Control of Unknown Affine Nonlinear Discrete-Time Systems

On Mixed Data and Event Driven Design for Adaptive-Critic-Based Nonlinear $h_{\infty}$ Control

Data-Driven Solutions to Mixed $h_{2}/h_{\infty}$ Control: A Hamilton-Inequality-Driven Reinforcement Learning Approach.

Data-driven $H_{\infty}$ predictive control for constrained systems: a Lagrange duality approcah

Model-Free $δ$-Policy Iteration Based on Damped Newton Method for Nonlinear Continuous-Time H$\infty$ Tracking Control

Synergetic Learning for Unknown Nonlinear H∞ Control Using Neural Networks.

$$H_\infty $$ Control Using Reinforcement Learning

Learning-Based Nonlinear $H^\infty$ Control via Game-Theoretic Differential Dynamic Programming

Data-Driven Learning for <italic>H</italic><inline-formula><tex-math notation="LaTeX">$_{\infty }$</tex-math></inline-formula> Control of Adaptive Cruise Control Systems

Event-driven H ∞ control with critic learning for nonlinear systems

$H_{\infty}$ Control for Interconnected Systems with Unknown System Dynamics: A Two-Stage Reinforcement Learning Method

Data-based Approximate Policy Iteration for Affine Nonlinear Continuous-Time Optimal Control Design.

Data-Driven &Lt;inline-Formula&gt; &Lt;tex-Math Notation="latex"&gt;$h_\infty$ &Lt;/tex-Math&gt;&lt;/inline-formula&gt; Control for Nonlinear Distributed Parameter Systems