Moderate deviations for empirical measures for nonhomogeneous Markov chains

Mingzhou Xu,Kun Cheng

DOI: https://doi.org/10.48550/arXiv.2010.07501

2020-10-15

Abstract:We prove that moderate deviations for empirical measures for countable nonhomogeneous Markov chains hold under the assumption of uniform convergence of transition probability matrices for countable nonhomogeneous Markov chains in Cesàro sense.

Probability,Statistics Theory

What problem does this paper attempt to address?

Central Limit Theorem and Moderate deviation for nonhomogenenous Markov chains

Mingzhou Xu,Yunzheng Ding,Yongzheng Zhou

DOI: https://doi.org/10.48550/arXiv.2010.06790

2020-10-14

Abstract:Our purpose is to prove central limit theorem for countable nonhomogeneous Markov chain under the condition of uniform convergence of transition probability matrices for countable nonhomogeneous Markov chain in Cesàro sense. Furthermore, we obtain a corresponding moderate deviation theorem for countable nonhomogeneous Markov chain by Gärtner-Ellis theorem and exponential equivalent method.

Probability,Statistics Theory
The Invariance Principle for Some Class of Markov Chains

ZY Lin

DOI: https://doi.org/10.4213/tvp615

2000-01-01

Theory of Probability and Its Applications

Abstract:This work is devoted to the invariance principle for homogeneous Markov chains with a finite state space. The proof is based on a comparison of the asymptotic behavior of sums of random variables constructed by Markov chains with the same transition matrix and various initial distributions.
Existence and Regularity of A Nonhomogeneous Transition Matrix under Measurability Conditions

Liuer Ye,Xianping Guo,Onésimo Hernández-Lerma

DOI: https://doi.org/10.1007/s10959-008-0163-9

2008-01-01

Journal of Theoretical Probability

Abstract:This paper is about the existence and regularity of the transition probability matrix of a nonhomogeneous continuous-time Markov process with a countable state space. A standard approach to prove the existence of such a transition matrix is to begin with a continuous (in t≥0) and conservative matrix Q(t)=[q ij (t)] of nonhomogeneous transition rates q ij (t) and use it to construct the transition probability matrix. Here we obtain the same result except that the q ij (t) are only required to satisfy a mild measurability condition, and Q(t) may not be conservative. Moreover, the resulting transition matrix is shown to be the minimum transition matrix, and, in addition, a necessary and sufficient condition for it to be regular is obtained. These results are crucial in some applications of nonhomogeneous continuous-time Markov processes, such as stochastic optimal control problems and stochastic games, and this was the main motivation for this work.
Geometric ergodicity of trans-dimensional Markov chain Monte Carlo algorithms

Qian Qin

2024-10-17

Abstract:This article studies the convergence properties of trans-dimensional MCMC algorithms when the total number of models is finite. It is shown that, for reversible and some non-reversible trans-dimensional Markov chains, under mild conditions, geometric convergence is guaranteed if the Markov chains associated with the within-model moves are geometrically ergodic. This result is proved in an $L^2$ framework using the technique of Markov chain decomposition. While the technique was previously developed for reversible chains, this work extends it to the point that it can be applied to some commonly used non-reversible chains. The theory herein is applied to reversible jump algorithms for three Bayesian models: a probit regression with variable selection, a Gaussian mixture model with unknown number of components, and an autoregression with Laplace errors and unknown model order.

Statistics Theory
Non-asymptotic Estimates for Markov Transition Matrices with Rigorous Error Bounds

De Huang,Xiangyuan Li

2024-08-12

Abstract:We establish non-asymptotic error bounds for the classical Maximal Likelihood Estimation of the transition matrix of a given Markov chain. Meanwhile, in the reversible case, we propose a new reversibility-preserving online Symmetric Counting Estimation of the transition matrix with non-asymptotic deviation bounds. Our analysis is based on a convergence study of certain Markov chains on the length-2 path spaces induced by the original Markov chain.

Statistics Theory,Probability
Large Deviations for Empirical Measures of Self-Interacting Markov Chains

Amarjit Budhiraja,Adam Waterbury,Pavlos Zoubouloglou

2023-04-11

Abstract:Let $\Delta^o$ be a finite set and, for each probability measure $m$ on $\Delta^o$, let $G(m)$ be a transition probability kernel on $\Delta^o$. Fix $x_0 \in \Delta^o$ and consider the chain $\{X_n, \; n \in \mathbb{N}_0\}$ of $\Delta^o$-valued random variables such that $X_0=x$, and given $X_0, \ldots , X_n$, the conditional distribution of $X_{n+1}$ is $G(L^{n+1})(X_n, \cdot)$, where $L^{n+1} = \frac{1}{n+1} \sum_{i=0}^{n} \delta_{X_i}$ is the empirical measure at instant $n$. Under conditions on $G$ we establish a large deviation principle for the empirical measure sequence $\{L^n, \; n \in \mathbb{N}\}$. As one application of this result we obtain large deviation asymptotics for the Aldous-Flannery-Palacios (1988) approximation scheme for quasistationary distributions of irreducible finite state Markov chains. The conditions on $G$ cover various other models of reinforced stochastic evolutions as well, including certain vertex reinforced and edge reinforced random walks and a variant of the PageRank algorithm. The particular case where $G(m)$ does not depend on $m$ corresponds to the classical results of Donsker and Varadhan (1975) on large deviations of empirical measures of Markov processes. However, unlike this classical setting, for the general self-interacting models considered here, the rate function takes a very different form; it is typically non-convex and is given through a dynamical variational formula with an infinite horizon discounted objective function.

Probability,Optimization and Control
THE EXPONENTIAL CONVERGENCE RATE OF THE SHANNON ENTROPY RATE OF COUNTABLE HOMOGENEOUS MARKOV CHAINS

HAO Ruili,YANG Weiguo,LI Rui

2010-01-01

Abstract:In this paper,the convergence rate of the Shannon entropy rate of certain countable homogeneous Markov chain is investigated by using Markov inequality and some properties of expectation,strong ergodicity and delta coefficient,in which the technique in Chang is applied.The obtained results generalize the corresponding results given by Chang.
Large Deviations for Markov Chains and Spectrum of Matrices

Jinwen Chen

DOI: https://doi.org/10.1360/n012019-00019

2019-01-01

Abstract:This paper deals with large deviations and related problems for Markov chains with countable state spaces. By comparing with some dominating chains that satisfy the large deviation principle, we weaken the requirement for irreducibility to get the large deviation principle. In this way we can also characterize the stationary distributions of the chains via the large deviation rate functions. In particular, we prove that any finite Markov chain satisfies the large deviation principle. The global and local minimums ofthe rate function are shown to be closely related to the principal eigenvalues and the corresponding left and right eigenfunctions of the transition matrix.
Large and Moderate Deviations for Empirical Density Fields of Stochastic Seir Epidemics with Vertex-Dependent Transition Rates

Xiaofeng Xue,Xueting Yin

DOI: https://doi.org/10.1007/s11118-024-10133-9

2024-03-12

Potential Analysis

Abstract:In this paper, we are concerned with stochastic susceptible-exposed-infected-removed epidemics on complete graphs with vertex-dependent transition rates. Large and moderate deviations of empirical density fields of our models are given. Proofs of our main results utilize exponential martingale strategies. In the proof of the moderate deviation principle, we introduce an iteration approach to check the exponential tightness of scaled density fields of our processes. As an application of our main results, moderate deviations of a family of hitting times of our processes are also given.

mathematics
Hoeffding's Inequality for Markov Chains under Generalized Concentrability Condition

Hao Chen,Abhishek Gupta,Yin Sun,Ness Shroff

2023-10-05

Abstract:This paper studies Hoeffding's inequality for Markov chains under the generalized concentrability condition defined via integral probability metric (IPM). The generalized concentrability condition establishes a framework that interpolates and extends the existing hypotheses of Markov chain Hoeffding-type inequalities. The flexibility of our framework allows Hoeffding's inequality to be applied beyond the ergodic Markov chains in the traditional sense. We demonstrate the utility by applying our framework to several non-asymptotic analyses arising from the field of machine learning, including (i) a generalization bound for empirical risk minimization with Markovian samples, (ii) a finite sample guarantee for Ployak-Ruppert averaging of SGD, and (iii) a new regret bound for rested Markovian bandits with general state space.

Machine Learning,Probability
A large deviation principle for the empirical measures of Metropolis-Hastings chains

Federica Milinanni,Pierre Nyquist

2023-08-14

Abstract:To sample from a given target distribution, Markov chain Monte Carlo (MCMC) sampling relies on constructing an ergodic Markov chain with the target distribution as its invariant measure. For any MCMC method, an important question is how to evaluate its efficiency. One approach is to consider the associated empirical measure and how fast it converges to the stationary distribution of the underlying Markov process. Recently, this question has been considered from the perspective of large deviation theory, for different types of MCMC methods, including, e.g., non-reversible Metropolis-Hastings on a finite state space, non-reversible Langevin samplers, the zig-zag sampler, and parallell tempering. This approach, based on large deviations, has proven successful in analysing existing methods and designing new, efficient ones. However, for the Metropolis-Hastings algorithm on more general state spaces, the workhorse of MCMC sampling, the same techniques have not been available for analysing performance, as the underlying Markov chain dynamics violate the conditions used to prove existing large deviation results for empirical measures of a Markov chain. This also extends to methods built on the same idea as Metropolis-Hastings, such as the Metropolis-Adjusted Langevin Method or ABC-MCMC. In this paper, we take the first steps towards such a large-deviations based analysis of Metropolis-Hastings-like methods, by proving a large deviation principle for the the empirical measures of Metropolis-Hastings chains. In addition, we characterize the rate function and its properties in terms of the acceptance- and rejection-part of the Metropolis-Hastings dynamics.

Probability,Computation
Cutoff for mixtures of permuted Markov chains: reversible case

Bastien Dubail

2024-02-08

Abstract:We investigate the mixing properties of a model of reversible Markov chains in random environment, which notably contains the simple random walk on the superposition of a deterministic graph and a second graph whose vertex set has been permuted uniformly at random. It generalizes in particular a result of Hermon, Sly and Sousi, who proved the cutoff phenomenon at entropic time for the simple random walk on a graph with an added uniform matching. Under mild assumptions on the base Markov chains, we prove that with high probability the resulting chain exhibits the cutoff phenomenon at entropic time log n/h, h being some constant related to the entropy of the chain. We note that the results presented here are the consequence of a work conducted for a more general model that does not assume reversibility, which will be the object of a companion paper. Thus, most of our proofs do not actually require reversibility, which constitutes an important technical contribution. Finally, our argument relies on a novel concentration result for "low-degree" functions on the symmetric group, established specifically for our purpose but which could be of independent interest.

Probability
A large deviation principle for the empirical measures of Metropolis–Hastings chains

Federica Milinanni,Pierre Nyquist

DOI: https://doi.org/10.1016/j.spa.2023.104293

IF: 1.43

2024-01-04

Stochastic Processes and their Applications

Abstract:To sample from a given target distribution, Markov chain Monte Carlo (MCMC) sampling relies on constructing an ergodic Markov chain with the target distribution as its invariant measure. For any MCMC method, an important question is how to evaluate its efficiency. One approach is to consider the associated empirical measure and how fast it converges to the stationary distribution of the underlying Markov process. Recently, this question has been considered from the perspective of large deviation theory, for different types of MCMC methods, including, e.g., non-reversible Metropolis–Hastings on a finite state space, non-reversible Langevin samplers, the zig-zag sampler, and parallel tempering. This approach, based on large deviations, has proven successful in analysing existing methods and designing new, efficient ones. However, for the Metropolis–Hastings algorithm on more general state spaces, the workhorse of MCMC sampling, the same techniques have not been available for analysing performance, as the underlying Markov chain dynamics violate the conditions used to prove existing large deviation results for empirical measures of a Markov chain. This also extends to methods built on the same idea as Metropolis–Hastings, such as the Metropolis-Adjusted Langevin Method or ABC-MCMC. In this paper, we take the first steps towards such a large-deviations based analysis of Metropolis–Hastings-like methods, by proving a large deviation principle for the empirical measures of Metropolis–Hastings chains. In addition, we also characterize the rate function and its properties in terms of the acceptance- and rejection-part of the Metropolis–Hastings dynamics.

statistics & probability
Graph-combinatorial approach for large deviations of Markov chains

Giorgio Carugno,Pierpaolo Vivo,Francesco Coghi

DOI: https://doi.org/10.1088/1751-8121/ac79e6

2022-06-03

Abstract:We consider discrete-time Markov chains and study large deviations of the pair empirical occupation measure, which is useful to compute fluctuations of pure-additive and jump-type observables. We provide an exact expression for the finite-time moment generating function, which is split in cycles and paths contributions, and scaled cumulant generating function of the pair empirical occupation measure via a graph-combinatorial approach. The expression obtained allows us to give a physical interpretation of interaction and entropic terms, and of the Lagrange multipliers, and may serve as a starting point for sub-leading asymptotics. We illustrate the use of the method for a simple two-state Markov chain.

Statistical Mechanics,Probability
Convergence analysis of some multivariate Markov chains using stochastic monotonicity

Kshitij Khare,Nabanita Mukherjee

DOI: https://doi.org/10.1214/12-AAP856

2013-02-22

Abstract:We provide a nonasymptotic analysis of convergence to stationarity for a collection of Markov chains on multivariate state spaces, from arbitrary starting points, thereby generalizing results in [Khare and Zhou Ann. Appl. Probab. 19 (2009) 737-777]. Our examples include the multi-allele Moran model in population genetics and its variants in community ecology, a generalized Ehrenfest urn model and variants of the Polya urn model. It is shown that all these Markov chains are stochastically monotone with respect to an appropriate partial ordering. Then, using a generalization of the results in [Diaconis, Khare and Saloff-Coste Sankhya 72 (2010) 45-76] and [Wilson Ann. Appl. Probab. 14 (2004) 274-325] (for univariate totally ordered spaces) to multivariate partially ordered spaces, we obtain explicit nonasymptotic bounds for the distance to stationarity from arbitrary starting points. In previous literature, bounds, if any, were available only from special starting points. The analysis also works for nonreversible Markov chains, and allows us to analyze cases of the multi-allele Moran model not considered in [Khare and Zhou Ann. Appl. Probab. 19 (2009) 737-777].

Probability
Hoeffding's lemma for Markov Chains and its applications to statistical learning

Jianqing Fan,Bai Jiang,Qiang Sun

DOI: https://doi.org/10.48550/arXiv.1802.00211

2018-02-01

Statistics Theory

Abstract:We extend Hoeffding's lemma to general-state-space and not necessarily reversible Markov chains. Let $\{X_i\}_{i \ge 1}$ be a stationary Markov chain with invariant measure $\pi$ and absolute spectral gap $1-\lambda$, where $\lambda$ is defined as the operator norm of the transition kernel acting on mean zero and square-integrable functions with respect to $\pi$. Then, for any bounded functions $f_i: x \mapsto [a_i,b_i]$, the sum of $f_i(X_i)$ is sub-Gaussian with variance proxy $\frac{1+\lambda}{1-\lambda} \cdot \sum_i \frac{(b_i-a_i)^2}{4}$. This result differs from the classical Hoeffding's lemma by a multiplicative coefficient of $(1+\lambda)/(1-\lambda)$, and simplifies to the latter when $\lambda = 0$. The counterpart of Hoeffding's inequality for Markov chains immediately follows. Our results assume none of countable state space, reversibility and time-homogeneity of Markov chains and cover time-dependent functions with various ranges. We illustrate the utility of these results by applying them to six problems in statistics and machine learning.
A class of strong deviation theorems for the sequence of real valued random variables with respect to continuous-state non-homogeneous Markov chains

Meng-di Zhao,Zhi-yan Shi,Wei-guo Yang,Bei Wang

DOI: https://doi.org/10.1080/03610926.2020.1734838

2020-03-03

Abstract:In this paper, we consider the asymptotic behavior of the log-likelihood ratio as a measure of the deviation between a sequence of real valued random variables and a continuous-state non-homogeneous Markov chains. Then, by using the supermartingale limit theorem, we establish a class of strong deviation theorems for bivariate functions of the sequence of real valued random variables which are associated with the continuous-state non-homogeneous Markov chains.
Cutoff for mixtures of permuted Markov chains: general case

Bastien Dubail

2024-02-06

Abstract:We investigate the mixing properties of a finite Markov chain in random environment defined as a mixture of a deterministic chain and a chain whose state space has been permuted uniformly at random. This work is the counterpart of a companion paper where we focused on a reversible model, which allowed for a few simplifications in the proof. We consider here the general case. Under mild assumptions on the base Markov chains, we prove that with high probability the resulting chain exhibits the cutoff phenomenon at entropic time $\log n / h$, $h$ being some constant related to the entropy of the chain, when the chain is started from a typical state. However contrary to the reversible case uniform cutoff at entropic time does not hold, as we provide an example where the worst-case mixing time has at least polylogarithmic order. We also provide a polylogarithmic upper bound on the worst-case mixing time, which in fact plays a crucial role in deriving the main result for typical states. Incidentally, our proof gives a clear picture of when to expect uniform cutoff at entropic time: it appears as the consequence of a uniform transience property for the covering Markov chain used throughout the proof, which lies on an infinite state space and projects back onto the initial chain.

Probability
Deviation inequalities, Moderate deviations and some limit theorems for bifurcating Markov chains with application

Valère Bitseki Penda,Hacène Djellout,Arnaud Guillin

DOI: https://doi.org/10.1214/13-AAP921

2014-01-17

Abstract:First, under a geometric ergodicity assumption, we provide some limit theorems and some probability inequalities for the bifurcating Markov chains (BMC). The BMC model was introduced by Guyon to detect cellular aging from cell lineage, and our aim is thus to complete his asymptotic results. The deviation inequalities are then applied to derive first result on the moderate deviation principle (MDP) for a functional of the BMC with a restricted range of speed, but with a function which can be unbounded. Next, under a uniform geometric ergodicity assumption, we provide deviation inequalities for the BMC and apply them to derive a second result on the MDP for a bounded functional of the BMC with a larger range of speed. As statistical applications, we provide superexponential convergence in probability and deviation inequalities (for either the Gaussian setting or the bounded setting), and the MDP for least square estimators of the parameters of a first-order bifurcating autoregressive process.

Probability
Convergence Bounds for Monte Carlo Markov Chains

Qian Qin

2024-09-23

Abstract:This review paper, written for the second edition of the Handbook of Markov Chain Monte Carlo, provides an introduction to the study of convergence analysis for Markov chain Monte Carlo (MCMC), aimed at researchers new to the field. We focus on methods for constructing bounds on the distance between the distribution of a Markov chain at a given time and its stationary distribution. Two widely-used approaches are explored: the coupling method and the L2 theory of Markov chains. For the latter, we emphasize techniques based on conductance and isoperimetric inequalities. Additionally, we briefly discuss strategies for identifying slow convergence in Markov chains.

Statistics Theory,Probability

Moderate deviations for empirical measures for nonhomogeneous Markov chains

Central Limit Theorem and Moderate deviation for nonhomogenenous Markov chains

The Invariance Principle for Some Class of Markov Chains

Existence and Regularity of A Nonhomogeneous Transition Matrix under Measurability Conditions

Geometric ergodicity of trans-dimensional Markov chain Monte Carlo algorithms

Non-asymptotic Estimates for Markov Transition Matrices with Rigorous Error Bounds

Large Deviations for Empirical Measures of Self-Interacting Markov Chains

THE EXPONENTIAL CONVERGENCE RATE OF THE SHANNON ENTROPY RATE OF COUNTABLE HOMOGENEOUS MARKOV CHAINS

Large Deviations for Markov Chains and Spectrum of Matrices

Large and Moderate Deviations for Empirical Density Fields of Stochastic Seir Epidemics with Vertex-Dependent Transition Rates

Hoeffding's Inequality for Markov Chains under Generalized Concentrability Condition

A large deviation principle for the empirical measures of Metropolis-Hastings chains

Cutoff for mixtures of permuted Markov chains: reversible case

A large deviation principle for the empirical measures of Metropolis–Hastings chains

Graph-combinatorial approach for large deviations of Markov chains

Convergence analysis of some multivariate Markov chains using stochastic monotonicity

Hoeffding's lemma for Markov Chains and its applications to statistical learning

A class of strong deviation theorems for the sequence of real valued random variables with respect to continuous-state non-homogeneous Markov chains

Cutoff for mixtures of permuted Markov chains: general case

Deviation inequalities, Moderate deviations and some limit theorems for bifurcating Markov chains with application

Convergence Bounds for Monte Carlo Markov Chains