Abstract:This paper considers the speed of convergence (mixing) of a finite Markov kernel $P$ with respect to the Kullback-Leibler divergence (entropy). Given a Markov kernel one defines either a discrete-time Markov chain (with the $n$-step transition kernel given by the matrix power $P^n$) or a continuous-time Markov process (with the time-$t$ transition kernel given by $e^{t(P-\mathrm{Id})}$). The contraction of entropy for $n=1$ or $t=0+$ are characterized by the famous functional inequalities, the strong data processing inequality (SDPI) and the modified log-Sobolev inequality (MLSI), respectively. When $P=KK^*$ is written as the product of a kernel and its adjoint, one could also consider the ``half-step'' contraction, which is the SDPI for $K$, while the ``full-step'' contraction refers to the SDPI for $P$. The work [DMLM03] claimed that these contraction coefficients (half-step, full-step, and continuous-time) are generally within a constant factor of each other. We disprove this and related conjectures by working out a number of different counterexamples. In particular, we construct (a) a continuous-time Markov process that contracts arbitrarily faster than its discrete-time counterpart; and (b) a kernel $P$ such that $P^{m+1}$ contracts arbitrarily better than $P^m$. Hence, our main conclusion is that the four standard inequalities comparing five common notions of entropy and variance contraction are generally not improvable. In the process of analyzing the counterexamples, we survey and sharpen the tools for bounding the contraction coefficients and characterize properties of extremizers of the respective functional inequalities. As our examples range from Bernoulli-Laplace model, random walks on graphs, to birth-death chains, the paper is also intended as a tutorial on computing MLSI, SDPI and other constants for these types of commonly occurring Markov chains.

Rates of Convergence for Chains of Expansive Markov Operators

Some Limit Properties of Markov Chains Induced by Stochastic Recursive Algorithms

Probabilistic Contraction Analysis of Iterated Random Operators

Convergence Rates of Markov Chains on Spaces of Partitions

Quantitative contraction rates for Markov chains on general state spaces

Path convergence of Markov chains on large graphs

Quantitative Convergence Rates for Stochastically Monotone Markov Chains

Deep Learning for Computing Convergence Rates of Markov Chains

On Convergence Rate Bounds for a Class of Nonlinear Markov Chains

Asymptotic rates of convergence for compact reversible Markov chains

Concentration of measure and mixing for Markov chains

Markov chains and mappings of distributions on compact spaces II: Numerics and Conjectures

Convergence of local times of stochastic processes associated with resistance forms

Markov chains and mappings of distributions on compact spaces

Entropy Contractions in Markov Chains: Half-Step, Full-Step and Continuous-Time

Lower bounds on the rate of convergence for accept-reject-based Markov chains in Wasserstein and total variation distances

Rate of Convergence of Space Time Approximations for Stochastic Evolution Equations

Wasserstein Convergence Rate for Empirical Measures of Markov Processes

Convergence to stable laws for a class of multidimensional stochastic recursions

Rates of Convergence in the Central Limit Theorem for Markov Chains, with an Application to TD Learning