Abstract:This paper is a survey of various proofs of the so called {\em fundamental theorem of Markov chains}: every ergodic Markov chain has a unique positive stationary distribution and the chain attains this distribution in the limit independent of the initial distribution the chain started with. As Markov chains are stochastic processes, it is natural to use probability based arguments for proofs. At the same time, the dynamics of a Markov chain is completely captured by its initial distribution, which is a vector, and its transition probability matrix. Therefore, arguments based on matrix analysis and linear algebra can also be used. The proofs discussed below use one or the other of these two types of arguments, except in one case where the argument is graph theoretic. Appropriate credits to the various proofs are given in the main text. Our first proof is entirely elementary, and yet the proof is also quite simple. The proof also suggests a mixing time bound, which we prove, but this bound in many cases will not be the best bound. One approach in proving the fundamental theorem breaks the proof in two parts: (i) show the existence of a unique positive stationary distribution for irreducible Markov chains, and (ii) assuming that an ergodic chain does have a stationary distribution, show that the chain will converge in the limit to that distribution irrespective of the initial distribution. For (i), we survey two proofs, one uses probability arguments, and the other uses graph theoretic arguments. For (ii), first we give a coupling based proof (coupling is a probability based technique), the other uses matrix analysis. Finally, we give a proof of the fundamental theorem using only linear algebra concepts.

Markov chains and mappings of distributions on compact spaces

Markov chains and mappings of distributions on compact spaces II: Numerics and Conjectures

Markov chains: Structure and applications

The Markov chain asymptotics of random mapping graphs

Markov chains revisited

Stationary measures of continuous time Markov chains with applications to stochastic reaction networks

Markov Chains Approximations of jump-Diffusion Quantum Trajectories

Rates of Convergence for Chains of Expansive Markov Operators

On regularity of functions of Markov chains

Markov tail chains

Some properties of Markov chains on the free group $\mathbb F_2$

On invariant distributions of Feller Markov chains with applications to dynamical systems with random switching

Topological dynamics of Markov multi-maps of the interval

Chain of Log-Concave Markov Chains

Various proofs of the Fundamental Theorem of Markov Chains

Geometry and Duality of Alternating Markov Chains

A Tutorial on the Spectral Theory of Markov Chains

Stationary entrance chains and applications to random walks

Markov chains on trees: almost lower and upper directed cases

Constructing Markov chains with given dependence and marginal stationary distributions

Concentration of measure and mixing for Markov chains