Abstract:A divide-and-conquer approach to analyzing Markov chains (MCs) is not utilized as widely as it could be, despite its potential benefits. One primary reason for this is the fact that most MC decomposition approaches involve a complex and inflexible methodology: decomposed subchains must be disjoint, transition rates of these decomposed subchains must be altered in a way tailored to the particular MC model, and the procedure to aggregate suchains needs to incorporate a nonlinear normalization constraint, complicating the analytical expression of performance measures. In contrast, we propose a versatile yet simple decomposition method for continuous time MCs based on the total expectation theorem. Leveraging the properties of this theorem, our method has great flexibility in the choice of subchains, and the procedure to obtain expected values of interest is simply a linear summation of subchains' properties, which is not affected by the normalization constraint. We prove that to maintain the correct distribution of decomposed subchains one may use our novel termination scheme, a modification of transition rates, that ensures partial flow conservation at boundary states. This termination scheme is applicable to MCs with any structure, since the scheme depends only on the boundary-state distribution, not on the structure of the MCs. To demonstrate the generality and capability of our method, we analytically solve various models, such as a congestion-based staffing queue and a Markov-modulated Mt/Mt/1 queue. As not all systems admit an analytical solution, we complement this analysis with numerical studies of MCs with various sizes using the algorithm based on our method.

Dynamical Decomposition of Markov Processes Without Detailed Balance

Flow Decomposition Reveals Dynamical Structure of Markov Process

Thermodynamics of Markov Processes with Non-extensive Entropy and Free Energy

Markov Chain Decomposition Based On Total Expectation Theorem

SDE Decomposition and A-type Stochastic Interpretation in Nonequilibrium Processes

Mixed Monotone Decomposition of Dynamical Systems with Application

Open Markov processes: A compositional perspective on non-equilibrium steady states in biology

Stability Of Circulation Decompositions And Self-Organization Phenomena

Action Principle and Dynamic Ensemble Theory for Non-equilibrium Markov Chains

Decomposition of Quantum Markov Chains and Its Applications

Optimal Data-Driven Estimation of Generalized Markov State Models for Non-Equilibrium Dynamics.

A New Type of Discrete Self-Decomposability and Its Application to Continuous-Time Markov Processes for Modeling Count Data Time Series

Work Relations Connecting Nonequilibrium Steady States Without Detailed Balance.

A Note on the Stability of Monotone Markov Chains

Feller Transition Functions, Resolvent Decomposition Theorems, and Their Application in Unstable Denumerable Markov Processes

State Space Decomposition for Nonautonomous Dynamical Systems

Elementary Bounds On Mixing Times for Decomposable Markov Chains

Operator-Theoretical Treatment of Ergodic Theorem and Its Application to Dynamic Models in Economics

Reversibility and Entropy Production of Inhomogeneous Markov Chains

Non-equilibrium Stochastic Dynamics

Nonequilibrium steady-state dynamics of Markov processes on graphs