Abstract:This paper is a survey of various proofs of the so called {\em fundamental theorem of Markov chains}: every ergodic Markov chain has a unique positive stationary distribution and the chain attains this distribution in the limit independent of the initial distribution the chain started with. As Markov chains are stochastic processes, it is natural to use probability based arguments for proofs. At the same time, the dynamics of a Markov chain is completely captured by its initial distribution, which is a vector, and its transition probability matrix. Therefore, arguments based on matrix analysis and linear algebra can also be used. The proofs discussed below use one or the other of these two types of arguments, except in one case where the argument is graph theoretic. Appropriate credits to the various proofs are given in the main text. Our first proof is entirely elementary, and yet the proof is also quite simple. The proof also suggests a mixing time bound, which we prove, but this bound in many cases will not be the best bound. One approach in proving the fundamental theorem breaks the proof in two parts: (i) show the existence of a unique positive stationary distribution for irreducible Markov chains, and (ii) assuming that an ergodic chain does have a stationary distribution, show that the chain will converge in the limit to that distribution irrespective of the initial distribution. For (i), we survey two proofs, one uses probability arguments, and the other uses graph theoretic arguments. For (ii), first we give a coupling based proof (coupling is a probability based technique), the other uses matrix analysis. Finally, we give a proof of the fundamental theorem using only linear algebra concepts.

Lie geometry of 2x2 Markov matrices

Lie Markov models with purine/pyrimidine symmetry

The geometry of Gaussian double Markovian distributions

Geometric models for Lie--Hamilton systems on $\mathbb{R}^2$

Geometric ergodicity of trans-dimensional Markov chain Monte Carlo algorithms

On the algebra of equal-input matrices in time-inhomogeneous Markov flows

Markov invariants for phylogenetic rate matrices derived from embedded submodels

Various proofs of the Fundamental Theorem of Markov Chains

Models of representations for classical series of Lie algebras

$\mathrm{SL}_2$-like Properties of Matrices Over Noncommutative Rings and Generalizations of Markov Numbers

A Minkowski space embedding to understand Markov models dynamics

Application of the Gel'fand Matrix Method to the Missing Label Problem in Classical Kinematical Lie Algebras

Permutation invariant Gaussian 2-matrix models

One-sided Markov additive processes with lattice and non-lattice increments

Geometric Aspects of Data-Processing of Markov Chains

Non-asymptotic Estimates for Markov Transition Matrices with Rigorous Error Bounds

Building Mean Field State Transition Models Using The Generalized Linear Chain Trick and Continuous Time Markov Chain Theory

Spectrum of large random reversible Markov chains: two examples

Low-Rank Tensors for Multi-Dimensional Markov Models

Some Probabilistic Properties of General Topological Markov Chains