The Invariance Principle for Some Class of Markov Chains

ZY Lin

DOI: https://doi.org/10.4213/tvp615

2000-01-01

Theory of Probability and Its Applications

Abstract:This work is devoted to the invariance principle for homogeneous Markov chains with a finite state space. The proof is based on a comparison of the asymptotic behavior of sums of random variables constructed by Markov chains with the same transition matrix and various initial distributions.

What problem does this paper attempt to address?

On the almost sure invariance principle for dependent Bernoulli random variables

yang zhang,lixin zhang

DOI: https://doi.org/10.1016/j.spl.2015.09.008

IF: 0.718

2015-01-01

Statistics & Probability Letters

Abstract:We consider a sequence of dependent Bernoulli variables where the success probability of the trial conditional on the past history is a linear function of the mean number of successes achieved to that point. An almost sure invariance principle is established for the partial sum and we also generalize the model to a multi-dimensional case, extending the results of Heyde (2004), James et al. (2008) and Wu et al. (2012).
STRONG APPROXIMATION OF A MARKOV CHAIN

Zhengyan LIN,Lixin ZHANG,Siu CHEUNG,Wai CHAN

DOI: https://doi.org/10.3321/j.issn:1000-8134.2005.02.011

2005-01-01

Abstract:The strong approximations of a class of non-homogenous Markov chains by Wiener processes are considered. As an application, the strong consistency, the law of the iterated logarithm and the weak convergence are established for the Markov chain adaptive designs in clinical trials.
On the rate of convergence in the weak invariance principle for dependent random variables with applications to Markov chains

Ion Grama,Émile Le Page,Marc Peigné

DOI: https://doi.org/10.4064/cm134-1-1

2024-12-20

Abstract:We prove an invariance principle for non-stationary random processes and establish a rate of convergence under a new type of mixing condition. The dependence is exponentially decaying in the gap between the past and the future and is controlled by an assumption on the characteristic function of the finite dimensional increments of the process. The distinct feature of the new mixing condition is that the dependence increases exponentially in the dimension of the increments. The proposed mixing property is particularly suited for processes whose behavior can be described in terms of spectral properties of some related family of operators. Several examples are discussed. We also work out explicit expressions for the constants involved in the bounds. When applied to Markov chains our result specifies the dependence of the constants on the properties of the underlying Banach space and on the initial state of the chain.

Probability
Various proofs of the Fundamental Theorem of Markov Chains

Somenath Biswas

DOI: https://doi.org/10.48550/arXiv.2204.00784

2022-04-02

Abstract:This paper is a survey of various proofs of the so called {\em fundamental theorem of Markov chains}: every ergodic Markov chain has a unique positive stationary distribution and the chain attains this distribution in the limit independent of the initial distribution the chain started with. As Markov chains are stochastic processes, it is natural to use probability based arguments for proofs. At the same time, the dynamics of a Markov chain is completely captured by its initial distribution, which is a vector, and its transition probability matrix. Therefore, arguments based on matrix analysis and linear algebra can also be used. The proofs discussed below use one or the other of these two types of arguments, except in one case where the argument is graph theoretic. Appropriate credits to the various proofs are given in the main text. Our first proof is entirely elementary, and yet the proof is also quite simple. The proof also suggests a mixing time bound, which we prove, but this bound in many cases will not be the best bound. One approach in proving the fundamental theorem breaks the proof in two parts: (i) show the existence of a unique positive stationary distribution for irreducible Markov chains, and (ii) assuming that an ergodic chain does have a stationary distribution, show that the chain will converge in the limit to that distribution irrespective of the initial distribution. For (i), we survey two proofs, one uses probability arguments, and the other uses graph theoretic arguments. For (ii), first we give a coupling based proof (coupling is a probability based technique), the other uses matrix analysis. Finally, we give a proof of the fundamental theorem using only linear algebra concepts.

Probability,Data Structures and Algorithms
Generalized Jarzynski'S Equality in Inhomogeneous Markov Chains

Hao Ge,Min Qian

DOI: https://doi.org/10.1063/1.2722561

IF: 1.469

2007-01-01

Journal of Mathematical Physics

Abstract:A rigorous mathematical theory of generalized Jarzynski’s equality in inhomogeneous Markov chains is given. Then, we explain its physical meaning and applications through several previous work including the original works of Jarzynski [Phys. Rev. Lett. 78, 2690 (1997); Phys. Rev. E 56, 5018 (1997); J. Stat. Phys. 96, 415 (1999); J. Stat. Phys. 98, 77 (2000)] and Crooks [J. Stat. Phys. 90, 1481 (1998); Phys. Rev. E 60, 2721 (1999); Phys. Rev. E 61, 2361 (2000)], Hatano-Sasa equality [Phys. Rev. E 86, 3463 (2001)], and the Gibbs free energy differences in stoichiometric chemical systems. By the way, we also consider the denumerable case.
A note on the invariance principle of the product of sums of random variables

Li-Xin Zhang,Wei Huang

DOI: https://doi.org/10.1214/ECP.v12-1255

2007-01-01

Electronic Communications in Probability

Abstract:The central limit theorem for the product of sums of various random variables has been studied in a variety of settings. The purpose of this note is to show that this kind of result is a corollary of the invariance principle.
Reversibility and Entropy Production of Inhomogeneous Markov Chains

Hao Ge,Da-Quan Jiang,Min Qian

DOI: https://doi.org/10.1239/jap/1165505205

2006-01-01

Abstract:In this paper we introduce the concepts of instantaneous reversibility and instantaneous entropy production rate for inhomogeneous Markov chains with denumerable state spaces. The following statements are proved to be equivalent: the inhomogeneous Markov chain is instantaneously reversible; it is in detailed balance; its entropy production rate vanishes. In particular, for a time-periodic birth-death chain, which can be regarded as a simple version of a physical model (Brownian motors), we prove that its rotation number is 0 when it is instantaneously reversible or periodically reversible. Hence, in our model of Markov chains, the directed transport phenomenon of Brownian motors can occur only in nonequilibrium and irreversible systems.
A Zero-One Law for Markov Chains

Michael Grabchak,Isaac Sonin

DOI: https://doi.org/10.48550/arXiv.2011.04063

2020-11-09

Abstract:We prove an analog of the classical Zero-One Law for both homogeneous and nonhomogeneous Markov chains (MC). Its almost precise formulation is simple: given any event $A$ from the tail $\sigma$-algebra of MC $(Z_n)$, for large $n$, with probability near one, the trajectories of the MC are in states $i$, where $P(A|Z_n=i)$ is either near $0$ or near $1$. A similar statement holds for the entrance $\sigma$-algebra, when $n$ tends to $-\infty$. To formulate this second result, we give detailed results on the existence of nonhomogeneous Markov chains indexed by $\mathbb Z_-$ or $\mathbb Z$ in both the finite and countable cases. This extends a well-known result due to Kolmogorov. Further, in our discussion, we note an interesting dichotomy between two commonly used definitions of MCs.

Probability
THE STOCHASTIC MONOTONICITY OF INHOMOGENEOUS MARKOV PROCESSES

Mei-juan ZHANG,Ming ZHANG

DOI: https://doi.org/10.3969/j.issn.0255-7797.2017.04.018

2017-01-01

Journal of Mathematics

Abstract:In this paper, we study the stochastic monotonicity of inhomogeneous Markov processes. By using and improving the proof method for the stochastic monotonicity of ho-mogeneous Markov processes, we obtain the explicit criterion for stochastic monotonicity of inhomogeneous Markov processes. Further more, this article extends this sufficient criterion to the equivalent condition.
On obtaining sharp bounds of the rate of convergence for a class of continuous-time Markov chains

A.I. Zeifman,Y.A. Satin,K.M. Kiseleva

DOI: https://doi.org/10.1016/j.spl.2020.108730

2020-06-01

Abstract:<p>We study inhomogeneous continuous-time weakly ergodic Markov chains with a finite state space. We introduce the notion of a Markov chain with the regular structure of an infinitesimal matrix and study the sharp upper bounds on the rate of convergence for such class of Markov chains.</p>

statistics & probability
Bernstein-type Inequalities for Markov Chains and Markov Processes: A Simple and Robust Proof

De Huang,Xiangyuan Li

2024-08-15

Abstract:We establish a new Bernstein-type deviation inequality for general discrete-time Markov chains via an elementary approach. More robust than existing works in the literature, our result only requires the Markov chain to satisfy an iterated Poincaré inequality. Moreover, our method can be readily generalized to continuous-time Markov processes.

Probability
Stable limits for Markov chains via the Principle of Conditioning

Mohamed El Machkouri,Adam Jakubowski,Dalibor Volný

2018-08-14

Abstract:We study limit theorems for partial sums of instantaneous functions of a homogeneous Markov chain on a general state space. The summands are heavy-tailed and the limits are stable distributions. The conditions imposed on the transition operator $P$ of the Markov chain ensure that the limit is the same as if the summands were independent. Such a~scheme admits a physical interpretation, as given in Jara et al. (Ann. Appl. Probab., 19 (2009), 2270--2300).

Mathematics
Some limit theorems for the second-order Markov chains indexed by a general infinite tree with uniform bounded degree

Zhiyan Shi,Weiguo Yang,Lixin Tian,Qingpei Zang

DOI: https://doi.org/10.1186/1029-242X-2012-2

2012-01-01

Journal of Inequalities and Applications

Abstract:In this paper, we study a convergence theorem for a finite second-order Markov chain indexed by a general infinite tree with uniformly bounded degree. Meanwhile, the strong law of large numbers (LLN) and Shannon-McMillan theorem for a finite second-order Markov chain indexed by this tree are obtained. 2000 Mathematics Subject Classification: 60F15; 60J10.
Remarks on limit theorems for reversible Markov processes

Martial Longla

DOI: https://doi.org/10.48550/arXiv.1310.8239

2018-01-14

Abstract:We propose some backward-forward martingale decompositions for functions of reversible Markov chains. These decompositions are used to prove the functional CLT for reversible Markov chains with asymptotically linear variance of partial sums. We also provide a proof of the equivalence between asymptotic linearity of the variance and convergence of the integral of $1/(1-t)$ with respect to the associated spectral measure $\rho$. We also study the asymptotic behavior of linear processes having as innovations mean zero square integrable functions of stationary reversible Markov chains. We apply this study to several cases of reversible stationary Markov chains that arise in regression estimation.

Probability
Invariant Distributions in Nonlinear Markov Chains with an Aggregator: Theory and Applications

Bar Light

2024-11-02

Abstract:We study the properties of a subclass of stochastic processes called discrete-time nonlinear Markov chains with an aggregator. In these chains, the next period's distribution depends on both the current state and a real-valued function of the current distribution. For these chains, we provide conditions for the uniqueness of an invariant distribution that do not rely on typical contraction arguments. Instead, our approach leverages flexible monotonicity properties imposed on the nonlinear Markov kernel. We also provide existence results and introduce a straightforward computational method for finding the invariant distribution. We demonstrate the necessity of these monotonicity conditions in proving the uniqueness of an invariant distribution through simple examples. We apply our findings to analyze invariant distributions in strategic queueing systems, establish conditions under which a class of nonlinear equations in $\mathbb{R}^{n}$ has a unique solution, and examine the properties of stationary wealth distributions in large dynamic economies.

Probability
Statistical Methods in Markov Chains

Patrick Billingsley

DOI: https://doi.org/10.1214/aoms/1177705136

1961-03-01

The Annals of Mathematical Statistics

Abstract:This paper is an expository survey of the mathematical aspects of statistical inference as it applies to finite Markov chains, the problem being to draw inferences about the transition probabilities from one long, unbroken observation $\{x_1, x_2, \cdots, x_n\}$ on the chain. The topics covered include Whittle's formula, chi-square and maximum-likelihood methods, estimation of parameters, and multiple Markov chains. At the end of the paper it is briefly indicated how these methods can be applied to a process with an arbitrary state space or a continuous time parameter. Section 2 contains a simple proof of Whittle's formula; Section 3 provides an elementary and self-contained development of the limit theory required for the application of chi-square methods to finite chains. In the remainder of the paper, the results are accompanied by references to the literature, rather than by complete proofs. As is usual in a review paper, the emphasis reflects the author's interests. Other general accounts of statistical inference on Markov processes will be found in Grenander [53], Bartlett [9] and [10], Fortet [35], and in my monograph [18]. I would like to thank Paul Meier for a number of very helpful discussions on the topics treated in this paper, particularly those of Section 3.
The Generalized Entropy Ergodic Theorem for Nonhomogeneous Markov Chains

Zhongzhi Wang,Weiguo Yang

DOI: https://doi.org/10.48550/arXiv.1501.03899

2015-01-16

Abstract:Let $(\xi_n)_{n=0}^\infty$ be a nonhomogeneous Markov chain taking values from finite state-space of $\mathbf{X}=\{1,2,\ldots,b\}$. In this paper, we will study the generalized entropy ergodic theorem with almost-everywhere and $\mathcal{L}_1$ convergence for nonhomogeneous Markov chains, which generalizes the corresponding classical results for the Markov chains.

Probability
Central Limit Theorem and Moderate deviation for nonhomogenenous Markov chains

Mingzhou Xu,Yunzheng Ding,Yongzheng Zhou

DOI: https://doi.org/10.48550/arXiv.2010.06790

2020-10-14

Abstract:Our purpose is to prove central limit theorem for countable nonhomogeneous Markov chain under the condition of uniform convergence of transition probability matrices for countable nonhomogeneous Markov chain in Cesàro sense. Furthermore, we obtain a corresponding moderate deviation theorem for countable nonhomogeneous Markov chain by Gärtner-Ellis theorem and exponential equivalent method.

Probability,Statistics Theory
Generalized Invariance Principles for Discrete-Time Stochastic Dynamical Systems

Shijie Zhou,Wei Lin,Xuerong Mao,Jianhong Wu

DOI: https://doi.org/10.1016/j.automatica.2022.110436

IF: 6.549

2023-01-01

IEEE Transactions on Automatic Control

Abstract:This article, based on the typical discrete-time semi-martingale convergence theorem, establishes several generalized versions of invariance principle for describing the long-term dynamical behaviors of discrete-time stochastic dynamical systems. These principles are suitable for investigating the dynamics in autonomous or non-autonomous systems and their applicability is demonstrated via using several representative examples. Particularly for autonomous systems, the established principle renders it possible to estimate the time when an orbit, initiating outside a particular bounded set, finally enters it. Furthermore, we provide a generalized version of discrete-time semi-martingale convergence theorem, and offer a counterexample to urge attentions to some delicate conditions that must be taken into account in the use of some version of convergence theorem.
Many strong limit theorems for non-homogeneous Markov chains indexed by a non-homogeneous tree

Shaohua Jin,Huitao Liu,Jian Liu,Yongxue Wang,Jian Zhang

DOI: https://doi.org/10.1109/ETCS.2010.346

2010-01-01

Abstract:The strong limit theorems is one of the central questions for studying in the International Probability theory. In this paper, many strong limit theorems for non-homogeneous Markov chains indexed by a non-homogeneous tree were obtained by constructing a non-negative martingale. © 2010 IEEE.

The Invariance Principle for Some Class of Markov Chains

On the almost sure invariance principle for dependent Bernoulli random variables

STRONG APPROXIMATION OF A MARKOV CHAIN

On the rate of convergence in the weak invariance principle for dependent random variables with applications to Markov chains

Various proofs of the Fundamental Theorem of Markov Chains

Generalized Jarzynski'S Equality in Inhomogeneous Markov Chains

A note on the invariance principle of the product of sums of random variables

Reversibility and Entropy Production of Inhomogeneous Markov Chains

A Zero-One Law for Markov Chains

THE STOCHASTIC MONOTONICITY OF INHOMOGENEOUS MARKOV PROCESSES

On obtaining sharp bounds of the rate of convergence for a class of continuous-time Markov chains

Bernstein-type Inequalities for Markov Chains and Markov Processes: A Simple and Robust Proof

Stable limits for Markov chains via the Principle of Conditioning

Some limit theorems for the second-order Markov chains indexed by a general infinite tree with uniform bounded degree

Remarks on limit theorems for reversible Markov processes

Invariant Distributions in Nonlinear Markov Chains with an Aggregator: Theory and Applications

Statistical Methods in Markov Chains

The Generalized Entropy Ergodic Theorem for Nonhomogeneous Markov Chains

Central Limit Theorem and Moderate deviation for nonhomogenenous Markov chains

Generalized Invariance Principles for Discrete-Time Stochastic Dynamical Systems

Many strong limit theorems for non-homogeneous Markov chains indexed by a non-homogeneous tree