The Smallest Eigenvalue for Reversible Markov Chains

XD Zhang

DOI: https://doi.org/10.1016/j.laa.2003.12.007

IF: 1.307

2004-01-01

Linear Algebra and its Applications

Abstract:We define a geometric quantity for reversible Markov chains and use it to prove lower and upper bounds for the smallest eigenvalue of such chains. Known bounds are discussed and compared with the new ones.

What problem does this paper attempt to address?

Upper Bound for the Non-Maximal Eigenvalues of Irreducible Nonnegative Matrices

Xiao-Dong Zhang,Rong Luo

DOI: https://doi.org/10.1023/a:1021771612003

2002-01-01

Czechoslovak Mathematical Journal

Abstract:We present a lower and an upper bound for the second smallest eigenvalue of Laplacian matrices in terms of the averaged minimal cut of weighted graphs. This is used to obtain an upper bound for the real parts of the non-maximal eigenvalues of irreducible nonnegative matrices. The result can be applied to Markov chains.
On the Smallest Eigenvalue of Grounded Laplacian Matrices

Mohammad Pirani,Shreyas Sundaram

DOI: https://doi.org/10.1109/tac.2015.2444191

IF: 6.549

2015-01-01

IEEE Transactions on Automatic Control

Abstract:We provide bounds on the smallest eigenvalue of grounded Laplacian matrices (which are obtained by removing certain rows and columns of the Laplacian matrix of a given graph). The gap between our upper and lower bounds depends on the ratio of the smallest and largest components of the eigenvector corresponding to the smallest eigenvalue of the grounded Laplacian. We provide a graph-theoretic bound on this ratio, and subsequently obtain a tight characterization of the smallest eigenvalue for certain classes of graphs. Specifically, for weighted Erdos-Renyi random graphs, we show that when a (sufficiently small) set S of rows and columns is removed from the Laplacian, and the probability p of adding an edge is sufficiently large, the smallest eigenvalue of the grounded Laplacian asymptotically almost surely approaches μw|S|p, where μw is the mean edge weight. We also show that for weighted random d-regular graphs with a single row and column removed, the smallest eigenvalue is Θ(1/n), where n is the number of nodes in the network. Our bounds have applications to the study of the convergence rate in consensus dynamics with stubborn or leader nodes.

automation & control systems,engineering, electrical & electronic
Lower Bound for the Second Smallest Eigenvalue of Directed Rooted Graph Laplacian

Huang Chao,Ye Xudong

2012-01-01

Abstract:A lower bound for the second smallest eigenvalue (SSE) of the unweighted Laplacian for an N-vertex directed rooted graph is obtained by obtaining the supremum of the scrambling constant of the (N - 1)-th power of the corresponding adjacency matrix. This supremum is actually achieved by the "N-layer Complete Graph" (NCG) defined in this paper, which implies that for directed rooted graphs that is unweighted for its directed edges, NCGs have the least connective topology in the sense of scrambling constant.
Spectral gap of nonreversible Markov chains

Sourav Chatterjee

2023-11-14

Abstract:We define the spectral gap of a Markov chain on a finite state space as the second-smallest singular value of the generator of the chain, generalizing the usual definition of spectral gap for reversible chains. We then define the relaxation time of the chain as the inverse of this spectral gap, and show that this relaxation time can be characterized, for any Markov chain, as the time required for convergence of empirical averages. This relaxation time is related to the Cheeger constant and the mixing time of the chain through inequalities that are similar to the reversible case, and the path argument can be used to get upper bounds. Several examples are worked out. An interesting finding from the examples is that the time for convergence of empirical averages in nonreversible chains can often be substantially smaller than the mixing time.

Probability,Combinatorics,Spectral Theory
Geometric Ergodicity and the Spectral Gap of Non-Reversible Markov Chains

Ioannis Kontoyiannis,Sean P. Meyn

DOI: https://doi.org/10.48550/arXiv.0906.5322

2009-06-30

Abstract:We argue that the spectral theory of non-reversible Markov chains may often be more effectively cast within the framework of the naturally associated weighted-$L_\infty$ space $L_\infty^V$, instead of the usual Hilbert space $L_2=L_2(\pi)$, where $\pi$ is the invariant measure of the chain. This observation is, in part, based on the following results. A discrete-time Markov chain with values in a general state space is geometrically ergodic if and only if its transition kernel admits a spectral gap in $L_\infty^V$. If the chain is reversible, the same equivalence holds with $L_2$ in place of $L_\infty^V$, but in the absence of reversibility it fails: There are (necessarily non-reversible, geometrically ergodic) chains that admit a spectral gap in $L_\infty^V$ but not in $L_2$. Moreover, if a chain admits a spectral gap in $L_2$, then for any $h\in L_2$ there exists a Lyapunov function $V_h\in L_1$ such that $V_h$ dominates $h$ and the chain admits a spectral gap in $L_\infty^{V_h}$. The relationship between the size of the spectral gap in $L_\infty^V$ or $L_2$, and the rate at which the chain converges to equilibrium is also briefly discussed.

Probability,Spectral Theory
Reversible Markov chains: variational representations and ordering

Chris Sherlock

DOI: https://doi.org/10.48550/arXiv.1809.01903

2018-09-06

Abstract:This pedagogical document explains three variational representations that are useful when comparing the efficiencies of reversible Markov chains: (i) the Dirichlet form and the associated variational representations of the spectral gaps; (ii) a variational representation of the asymptotic variance of an ergodic average; and (iii) the conductance, and the equivalence of a non-zero conductance to a non-zero right spectral gap.

Statistics Theory,Computation
Information Geometry of Reversible Markov Chains

Geoffrey Wolfer,Shun Watanabe

DOI: https://doi.org/10.1007/s41884-021-00061-7

2021-11-22

Information Geometry

Abstract:Abstract We analyze the information geometric structure of time reversibility for parametric families of irreducible transition kernels of Markov chains. We define and characterize reversible exponential families of Markov kernels, and show that irreducible and reversible Markov kernels form both a mixture family and, perhaps surprisingly, an exponential family in the set of all stochastic kernels. We propose a parametrization of the entire manifold of reversible kernels, and inspect reversible geodesics. We define information projections onto the reversible manifold, and derive closed-form expressions for the e-projection and m-projection, along with Pythagorean identities with respect to information divergence, leading to some new notion of reversiblization of Markov kernels. We show the family of edge measures pertaining to irreducible and reversible kernels also forms an exponential family among distributions over pairs. We further explore geometric properties of the reversible family, by comparing them with other remarkable families of stochastic matrices. Finally, we show that reversible kernels are, in a sense we define, the minimal exponential family generated by the m-family of symmetric kernels, and the smallest mixture family that comprises the e-family of memoryless kernels.

English Else
Non-asymptotic Estimates for Markov Transition Matrices with Rigorous Error Bounds

De Huang,Xiangyuan Li

2024-08-12

Abstract:We establish non-asymptotic error bounds for the classical Maximal Likelihood Estimation of the transition matrix of a given Markov chain. Meanwhile, in the reversible case, we propose a new reversibility-preserving online Symmetric Counting Estimation of the transition matrix with non-asymptotic deviation bounds. Our analysis is based on a convergence study of certain Markov chains on the length-2 path spaces induced by the original Markov chain.

Statistics Theory,Probability
Forbidden induced subgraphs for graphs and signed graphs with eigenvalues bounded from below

Zilin Jiang,Alexandr Polyanskii

2024-04-11

Abstract:The smallest eigenvalue of a graph is the smallest eigenvalue of its adjacency matrix. We show that the family of graphs with smallest eigenvalue at least $-\lambda$ can be defined by a finite set of forbidden induced subgraphs if and only if $\lambda < \lambda^*$, where $\lambda^* = \rho^{1/2} + \rho^{-1/2} \approx 2.01980$, and $\rho$ is the unique real root of $x^3 = x + 1$. This resolves a question raised by Bussemaker and Neumaier. As a byproduct, we find all the limit points of smallest eigenvalues of graphs, supplementing Hoffman's work on those limit points in $[-2, \infty)$.

Metric Geometry
Cutoff for mixtures of permuted Markov chains: reversible case

Bastien Dubail

2024-02-08

Abstract:We investigate the mixing properties of a model of reversible Markov chains in random environment, which notably contains the simple random walk on the superposition of a deterministic graph and a second graph whose vertex set has been permuted uniformly at random. It generalizes in particular a result of Hermon, Sly and Sousi, who proved the cutoff phenomenon at entropic time for the simple random walk on a graph with an added uniform matching. Under mild assumptions on the base Markov chains, we prove that with high probability the resulting chain exhibits the cutoff phenomenon at entropic time log n/h, h being some constant related to the entropy of the chain. We note that the results presented here are the consequence of a work conducted for a more general model that does not assume reversibility, which will be the object of a companion paper. Thus, most of our proofs do not actually require reversibility, which constitutes an important technical contribution. Finally, our argument relies on a novel concentration result for "low-degree" functions on the symmetric group, established specifically for our purpose but which could be of independent interest.

Probability
A Geometric Reduction Approach for Identity Testing of Reversible Markov Chains

Geoffrey Wolfer,Shun Watanabe

DOI: https://doi.org/10.48550/arXiv.2302.08059

2023-02-16

Probability

Abstract:We consider the problem of testing the identity of a reversible Markov chain against a reference from a single trajectory of observations. Employing the recently introduced notion of a lumping-congruent Markov embedding, we show that, at least in a mildly restricted setting, testing identity to a reversible chain reduces to testing to a symmetric chain over a larger state space and recover state-of-the-art sample complexity for the problem.
Tighter Bounds on the Expected Absorbing Time of Ungarian Markov Chains

Eric Shen

2024-05-20

Abstract:In 2023, Defant and Li defined the Ungarian Markov chain associated to a lattice $L$. This Markov chain has state space $L$, and from any state $x \in L$ transitions to the meet of $\{x\} \cup T$, where $T$ is a randomly selected subset of the elements of $L$ covered by $x$. They defined $\mathcal{E}(L)$ to be the expected number of steps until the maximal element $\hat{1}$ of $L$ transitions into the minimal element $\hat{0}$ in the Ungarian Markov chain, and investigated its asymptotics when $L$ is the weak order on $S_n$, or the $n$th Tamari lattice $\textrm{Tam}_n$. This paper resolves a conjecture of Defant and Li and makes partial progress towards another, by proving that $\mathcal{E}(S_n)$ is linear in $n$, and proving an $n^{1 - o(1)}$ lower bound on $\mathcal{E}(\textrm{Tam}_n)$.

Combinatorics,Probability
Entropy and curvature: beyond the Peres-Tetali conjecture

Pietro Caputo,Florentin Münch,Justin Salez

2024-02-09

Abstract:We study Markov chains with non-negative sectional curvature on finite metric spaces. Neither reversibility, nor the restriction to a particular combinatorial distance are imposed. In this level of generality, we prove that a 1-step contraction in the Wasserstein distance implies a 1-step contraction in relative entropy, by the same amount. Our result substantially strengthens a recent breakthrough of the second author, and has the advantage of being applicable to arbitrary scales. This leads to a time-varying refinement of the standard Modified Log-Sobolev Inequality (MLSI), which allows us to leverage the well-acknowledged fact that curvature improves at large scales. We illustrate this principle with several applications, including birth and death chains, colored exclusion processes, permutation walks, Gibbs samplers for high-temperature spin systems, and attractive zero-range dynamics. In particular, we prove a MLSI with constant equal to the minimal rate increment for the mean-field zero-range process, thereby answering a long-standing question.

Probability,Combinatorics,Differential Geometry
Geometric bounds on the fastest mixing Markov chain

Sam Olesker-Taylor,Luca Zanetti

DOI: https://doi.org/10.1007/s00440-023-01257-x

IF: 1.944

2024-02-01

Probability Theory and Related Fields

Abstract:In the Fastest Mixing Markov Chain problem, we are given a graph and desire the discrete-time Markov chain with smallest mixing time subject to having equilibrium distribution uniform on V and non-zero transition probabilities only across edges of the graph. It is well-known that the mixing time of the lazy random walk on G is characterised by the edge conductance of G via Cheeger's inequality: . Analogously, we characterise the fastest mixing time via a Cheeger-type inequality but for a different geometric quantity, namely the vertex conductance of G : . This characterisation forbids fast mixing for graphs with small vertex conductance. To bypass this fundamental barrier, we consider Markov chains on G with equilibrium distribution which need not be uniform, but rather only -close to uniform in total variation. We show that it is always possible to construct such a chain with mixing time . Finally, we discuss analogous questions for continuous-time and time-inhomogeneous chains.

statistics & probability
Geometric ergodicity of trans-dimensional Markov chain Monte Carlo algorithms

Qian Qin

2024-10-17

Abstract:This article studies the convergence properties of trans-dimensional MCMC algorithms when the total number of models is finite. It is shown that, for reversible and some non-reversible trans-dimensional Markov chains, under mild conditions, geometric convergence is guaranteed if the Markov chains associated with the within-model moves are geometrically ergodic. This result is proved in an $L^2$ framework using the technique of Markov chain decomposition. While the technique was previously developed for reversible chains, this work extends it to the point that it can be applied to some commonly used non-reversible chains. The theory herein is applied to reversible jump algorithms for three Bayesian models: a probit regression with variable selection, a Gaussian mixture model with unknown number of components, and an autoregression with Laplace errors and unknown model order.

Statistics Theory
Hoeffding's lemma for Markov Chains and its applications to statistical learning

Jianqing Fan,Bai Jiang,Qiang Sun

DOI: https://doi.org/10.48550/arXiv.1802.00211

2018-02-01

Statistics Theory

Abstract:We extend Hoeffding's lemma to general-state-space and not necessarily reversible Markov chains. Let $\{X_i\}_{i \ge 1}$ be a stationary Markov chain with invariant measure $\pi$ and absolute spectral gap $1-\lambda$, where $\lambda$ is defined as the operator norm of the transition kernel acting on mean zero and square-integrable functions with respect to $\pi$. Then, for any bounded functions $f_i: x \mapsto [a_i,b_i]$, the sum of $f_i(X_i)$ is sub-Gaussian with variance proxy $\frac{1+\lambda}{1-\lambda} \cdot \sum_i \frac{(b_i-a_i)^2}{4}$. This result differs from the classical Hoeffding's lemma by a multiplicative coefficient of $(1+\lambda)/(1-\lambda)$, and simplifies to the latter when $\lambda = 0$. The counterpart of Hoeffding's inequality for Markov chains immediately follows. Our results assume none of countable state space, reversibility and time-homogeneity of Markov chains and cover time-dependent functions with various ranges. We illustrate the utility of these results by applying them to six problems in statistics and machine learning.
Improved Estimation of Relaxation Time in Non-reversible Markov Chains

Geoffrey Wolfer,Aryeh Kontorovich

2023-08-04

Abstract:We show that the minimax sample complexity for estimating the pseudo-spectral gap $\gamma_{\mathsf{ps}}$ of an ergodic Markov chain in constant multiplicative error is of the order of $$\tilde{\Theta}\left( \frac{1}{\gamma_{\mathsf{ps}} \pi_{\star}} \right),$$ where $\pi_\star$ is the minimum stationary probability, recovering the known bound in the reversible setting for estimating the absolute spectral gap [Hsu et al., 2019], and resolving an open problem of Wolfer and Kontorovich [2019]. Furthermore, we strengthen the known empirical procedure by making it fully-adaptive to the data, thinning the confidence intervals and reducing the computational complexity. Along the way, we derive new properties of the pseudo-spectral gap and introduce the notion of a reversible dilation of a stochastic matrix.

Statistics Theory,Probability
Elementary Bounds On Mixing Times for Decomposable Markov Chains

Natesh S. Pillai,Aaron Smith

DOI: https://doi.org/10.48550/arXiv.1507.04028

2016-02-03

Abstract:Many finite-state reversible Markov chains can be naturally decomposed into "projection" and "restriction" chains. In this paper we provide bounds on the total variation mixing times of the original chain in terms of the mixing properties of these related chains. This paper is in the tradition of existing bounds on Poincare and log-Sobolev constants of Markov chains in terms of similar decompositions [JSTV04, MR02, MR06, MY09]. Our proofs are simple, relying largely on recent results relating hitting and mixing times of reversible Markov chains [PS13, Oli12]. We describe situations in which our results give substantially better bounds than those obtained by applying existing decomposition results and provide examples for illustration.

Probability
Combinatorial upper bounds for the smallest eigenvalue of a graph

Aryan Esmailpour,Sara Saeedi Madani,Dariush Kiani

2024-04-14

Abstract:Let $G$ be a graph, and let $\lambda(G)$ denote the smallest eigenvalue of $G$. First, we provide an upper bound for $\lambda(G)$ based on induced bipartite subgraphs of $G$. Consequently, we extract two other upper bounds, one relying on the average degrees of induced bipartite subgraphs and a more explicit one in terms of the chromatic number and the independence number of $G$. In particular, motivated by our bounds, we introduce two graph invariants that are of interest on their own. Finally, special attention goes to the investigation of the sharpness of our bounds in various classes of graphs as well as the comparison with an existing well-known upper bound.

Combinatorics
Systematic approaches to generate reversiblizations of Markov chains

Michael C.H. Choi,Geoffrey Wolfer

DOI: https://doi.org/10.1109/TIT.2023.3304685

2023-09-11

Abstract:Given a target distribution $\pi$ and an arbitrary Markov infinitesimal generator $L$ on a finite state space $\mathcal{X}$, we develop three structured and inter-related approaches to generate new reversiblizations from $L$. The first approach hinges on a geometric perspective, in which we view reversiblizations as projections onto the space of $\pi$-reversible generators under suitable information divergences such as $f$-divergences. With different choices of functions $f$, we not only recover nearly all established reversiblizations but also unravel and generate new reversiblizations. Along the way, we unveil interesting geometric results such as bisection properties, Pythagorean identities, parallelogram laws and a Markov chain counterpart of the arithmetic-geometric-harmonic mean inequality governing these reversiblizations. This further serves as motivation for introducing the notion of information centroids of a sequence of Markov chains and to give conditions for their existence and uniqueness. Building upon the first approach, we view reversiblizations as generalized means. In this second approach, we construct new reversiblizations via different natural notions of generalized means such as the Cauchy mean or the dual mean. In the third approach, we combine the recently introduced locally-balanced Markov processes framework and the notion of convex $*$-conjugate in the study of $f$-divergence. The latter offers a rich source of balancing functions to generate new reversiblizations.

Probability,Information Theory,Computation

The Smallest Eigenvalue for Reversible Markov Chains

Upper Bound for the Non-Maximal Eigenvalues of Irreducible Nonnegative Matrices

On the Smallest Eigenvalue of Grounded Laplacian Matrices

Lower Bound for the Second Smallest Eigenvalue of Directed Rooted Graph Laplacian

Spectral gap of nonreversible Markov chains

Geometric Ergodicity and the Spectral Gap of Non-Reversible Markov Chains

Reversible Markov chains: variational representations and ordering

Information Geometry of Reversible Markov Chains

Non-asymptotic Estimates for Markov Transition Matrices with Rigorous Error Bounds

Forbidden induced subgraphs for graphs and signed graphs with eigenvalues bounded from below

Cutoff for mixtures of permuted Markov chains: reversible case

A Geometric Reduction Approach for Identity Testing of Reversible Markov Chains

Tighter Bounds on the Expected Absorbing Time of Ungarian Markov Chains

Entropy and curvature: beyond the Peres-Tetali conjecture

Geometric bounds on the fastest mixing Markov chain

Geometric ergodicity of trans-dimensional Markov chain Monte Carlo algorithms

Hoeffding's lemma for Markov Chains and its applications to statistical learning

Improved Estimation of Relaxation Time in Non-reversible Markov Chains

Elementary Bounds On Mixing Times for Decomposable Markov Chains

Combinatorial upper bounds for the smallest eigenvalue of a graph

Systematic approaches to generate reversiblizations of Markov chains