Abstract:This paper is a survey of various proofs of the so called {\em fundamental theorem of Markov chains}: every ergodic Markov chain has a unique positive stationary distribution and the chain attains this distribution in the limit independent of the initial distribution the chain started with. As Markov chains are stochastic processes, it is natural to use probability based arguments for proofs. At the same time, the dynamics of a Markov chain is completely captured by its initial distribution, which is a vector, and its transition probability matrix. Therefore, arguments based on matrix analysis and linear algebra can also be used. The proofs discussed below use one or the other of these two types of arguments, except in one case where the argument is graph theoretic. Appropriate credits to the various proofs are given in the main text. Our first proof is entirely elementary, and yet the proof is also quite simple. The proof also suggests a mixing time bound, which we prove, but this bound in many cases will not be the best bound. One approach in proving the fundamental theorem breaks the proof in two parts: (i) show the existence of a unique positive stationary distribution for irreducible Markov chains, and (ii) assuming that an ergodic chain does have a stationary distribution, show that the chain will converge in the limit to that distribution irrespective of the initial distribution. For (i), we survey two proofs, one uses probability arguments, and the other uses graph theoretic arguments. For (ii), first we give a coupling based proof (coupling is a probability based technique), the other uses matrix analysis. Finally, we give a proof of the fundamental theorem using only linear algebra concepts.

Singular value distribution of dense random matrices with block Markovian dependence

Joint CLT for Several Random Sesquilinear Forms with Applications to Large-Dimensional Spiked Population Models

Spectral norm bounds for block Markov chain random matrices

Concentration Inequalities for Sums of Markov Dependent Random Matrices

A Matrix Chernoff Bound for Markov Chains and Its Application to Co-Occurrence Matrices

On limit theorem for the eigenvalues of product of two random matrices

Meeting times of Markov chains via singular value decomposition

Large deviations for the largest singular value of sparse non-Hermitian matrices

Spectrum of large random reversible Markov chains: two examples

Universality of the least singular value and singular vector delocalisation for Lévy non-symmetric random matrices

Non-asymptotic theory of random matrices: extreme singular values

Random matrices with row constraints and eigenvalue distributions of graph Laplacians

Large Deviations for Markov Chains and Spectrum of Matrices

Various proofs of the Fundamental Theorem of Markov Chains

Circular law for random matrices with exchangeable entries

On Random Matrices Arising in Deep Neural Networks: General I.I.D. Case

Analysis of The Limiting Spectral Distribution of Large Random Matrices of The Marčenko-Pastur Type

Large deviations of the largest eigenvalue of supercritical sparse Wigner matrices

Asymptotic Theory of Eigenvectors for Random Matrices with Diverging Spikes

Singularity degree of structured random matrices

Concentration Bounds for Co-occurrence Matrices of Markov Chains.