Abstract:The solution of coupled discrete-time Markovian jump Lyapunov matrix equations (CDMJLMEs) is important in stability analysis and controller design for Markovian jump linear systems. This paper presents a simple and effective iterative method to produce numerical solutions to this class of matrix equations. The gradient-based algorithm is developed from an optimization point of view. A necessary and sufficient condition guaranteeing the convergence of the algorithm is established. This condition shows that the algorithm always converges provided the CDMJLMEs have unique solutions which is evidently different from the existing results that converge conditionally. A simple sufficient condition which is easy to test is also provided. The optimal step size in the algorithm such that the convergence rate of the algorithm is maximized is given explicitly. It turns out that an upper bound of the convergence rate is bounded by a function of the condition number of the augmented coefficient matrix of the CDMJLMEs. Some parameters are introduced to the algorithm that will potentially reduce the condition number and thus increase the convergence rate of the algorithm. A numerical example is used to illustrate the efficiency of the proposed approach.

A Model-Free Iteration Algorithm for Markov Jump Linear Systems Based on Gauss-Seidel Method

Successive over Relaxation for Model-Free LQR Control of Discrete-Time Markov Jump Systems

General Lyapunov-Based Iterative Algorithm for Linear Quadratic Regulator Problem of Stochastic Systems with Markovian Jump

Asynchronous Static Output-Feedback Control of Markovian Jump Linear Systems

An Iterative Algorithm for Coupled Riccati Equations in Continuous-Time Markovian Jump Linear Systems

Finite-time L2−l∞ Tracking Control for Markov Jump Repeated Scalar Nonlinear Systems with Partly Usable Model Information

Convergence of Gradient-Based Iterative Solution of Coupled Markovian Jump Lyapunov Equations.

Cooperative Output Regulation Quadratic Control for Discrete-Time Heterogeneous Multiagent Markov Jump Systems

An Inversion-Free Iterative Algorithm with a Scalar Tuning Parameter for Coupled Riccati Matrix Equations Arising in LQ Optimal Control of Markov Jump Systems

Implicit Iterative Algorithms for Continuous Markovian Jump Lyapunov Equations

H∞$$ {h}_{\infty } $$ Optimal Output Tracking Control for Markov Jump Systems: A Reinforcement Learning‐based Approach

Asynchronous Static Output Feedback Control of Discrete-time Markov Jump Systems

Finite Iterative Algorithm for Solving Coupled Lyapunov Equations Appearing in Continuous-Time Markov Jump Linear Systems.

Hidden-Markov-Model-Based Asynchronous Filter Design of Nonlinear Markov Jump Systems in Continuous-Time Domain.

Model-free Value Iteration Algorithm for Continuous-time Stochastic Linear Quadratic Optimal Control Problems

New Iterative Algorithms for Solving Coupled Markovian Jump Lyapunov Equations

Jump Linear Quadratic Gaussian Problem for A Class of Nonhomogeneous Markov Jump Linear Systems

A Novel Iterative Algorithm for Solving Coupled Riccati Equations

Model-free optimal controller for discrete-time Markovian jump linear systems: A Q-learning approach

Stochastic LQ optimal control for Markov jumping systems with multiplicative noise using reinforcement learning

Positive Operator Based Iterative Algorithms for Solving Lyapunov Equations for Itô Stochastic Systems with Markovian Jumps