Abstract:We study continuous-time Markov chains on the non-negative integers under mild regularity conditions (in particular, the set of jump vectors is finite and both forward and backward jumps are possible). Based on the so-called flux balance equation, we derive an iterative formula for calculating stationary measures. Specifically, a stationary measure $\pi(x)$ evaluated at $x\in\mathbb{N}_0$ is represented as a linear combination of a few generating terms, similarly to the characterization of a stationary measure of a birth-death process, where there is only one generating term, $\pi(0)$. The coefficients of the linear combination are recursively determined in terms of the transition rates of the Markov chain. For the class of Markov chains we consider, there is always at least one stationary measure (up to a scaling constant). We give various results pertaining to uniqueness and non-uniqueness of stationary measures, and show that the dimension of the linear space of signed invariant measures is equal to the number of generating terms. A minimization problem is constructed in order to compute stationary measures numerically. Moreover, a heuristic linear approximation scheme is suggested for the same purpose by first approximating the generating terms. The correctness of the linear approximation scheme is justified in some special cases. Furthermore, a decomposition of the state space into different types of states (open and closed irreducible classes, and trapping, escaping and neutral states) is presented. Applications to stochastic reaction networks are well illustrated.

Regularity of Harmonic Functions for Some Markov Chains with Unbounded Range

On regularity of functions of Markov chains

The local time of the Markov processes of Ornstein–Uhlenbeck type

The Invariance Principle for Some Class of Markov Chains

The local properties of the Markov processes of Ornstein-Uhlenbeck type

Regularity Of Harmonic Maps Into Certain Homogeneous Spaces

Multivariate regular variation of heavy-tailed Markov chains

Hoeffding's lemma for Markov Chains and its applications to statistical learning

Markov chains in the domain of attraction of Brownian motion in cones

On invariant distributions of Feller Markov chains with applications to dynamical systems with random switching

On Regular Dirichlet Subspaces of H-1(I) and Associated Linear Diffusions

Stationary measures of continuous time Markov chains with applications to stochastic reaction networks

Spectral theoretic characterisation of Markov chain convergence

Markov Chains On Continuous State Space

The Generalized Entropy Ergodic Theorem for Nonhomogeneous Markov Chains

Exponential Stability of Matrix-Valued Markov Chains Via Nonignorable Periodic Data

On the Stability of Jump-Diffusions with Markovian Switching

Large Deviations for Markov Chains and Spectrum of Matrices

Markov chains: Structure and applications

Harmonic Maps on Domains with Piecewise Lipschitz Continuous Metrics

Pointwise Stabilization of Discrete-time Stationary Matrix-valued Markovian Chains