Abstract:This paper is a survey of various proofs of the so called {\em fundamental theorem of Markov chains}: every ergodic Markov chain has a unique positive stationary distribution and the chain attains this distribution in the limit independent of the initial distribution the chain started with. As Markov chains are stochastic processes, it is natural to use probability based arguments for proofs. At the same time, the dynamics of a Markov chain is completely captured by its initial distribution, which is a vector, and its transition probability matrix. Therefore, arguments based on matrix analysis and linear algebra can also be used. The proofs discussed below use one or the other of these two types of arguments, except in one case where the argument is graph theoretic. Appropriate credits to the various proofs are given in the main text. Our first proof is entirely elementary, and yet the proof is also quite simple. The proof also suggests a mixing time bound, which we prove, but this bound in many cases will not be the best bound. One approach in proving the fundamental theorem breaks the proof in two parts: (i) show the existence of a unique positive stationary distribution for irreducible Markov chains, and (ii) assuming that an ergodic chain does have a stationary distribution, show that the chain will converge in the limit to that distribution irrespective of the initial distribution. For (i), we survey two proofs, one uses probability arguments, and the other uses graph theoretic arguments. For (ii), first we give a coupling based proof (coupling is a probability based technique), the other uses matrix analysis. Finally, we give a proof of the fundamental theorem using only linear algebra concepts.

On the Limiting Probability Distribution of a Transition Probability Tensor

Finding stationary probability vector of a transition probability tensor arising from a higher order Markov chain

Higher Order Ergodic Markov Chains and First Passage Times

Limit Theorems of Power Variation for Time-Changed Stable Levy Processes

The Perturbation Bound for the Perron Vector of a Transition Probability Tensor

Triangular and Skew-Symmetric Splitting Method for Numerical Solutions of Markov Chains

Cyclic stationary probability distribution of second order markov chains and its applications

On the Uniqueness and Non-Uniqueness of the Positive Z-eigenvector for Transition Probability Tensors

A C-eigenvalue Problem for Tensors with Applications to Higher-Order Multivariate Markov Chains.

Limit Theorems for Dependent Bernoulli Variables with Statistical Inference

Non-asymptotic Estimates for Markov Transition Matrices with Rigorous Error Bounds

Limit Theorems and Phase Transitions in the Tensor Curie-Weiss Potts Model

Limit distribution of partial transposition of block random matrices

On exceedance times for some processes with dependent increments

Metastable Distributions of Semi-Markov Processes

Various proofs of the Fundamental Theorem of Markov Chains

A Universal Robust Limit Theorem for Nonlinear Lévy Processes under Sublinear Expectation

Large deviation for Gibbs probabilities at zero temperature and invariant idempotent probabilities for iterated function systems

Limiting Spectral Distribution of Random Birth-Death Q Matrices

Remarks on limit theorems for reversible Markov processes

Limit formulas for the normalized fundamental matrix of the northwest-corner truncation of Markov chains: Matrix-infinite-product-form solutions of block-Hessenberg Markov chains