A new criterion for recurrence of Markov chains with an infinitely countable set of states

Vyacheslav M. Abramov

2024-10-05

Abstract:For a class of irreducible Markov chains with an infinitely countable set of states, we establish a new verifiable necessary and sufficient condition for recurrence and transience. We show that if one of the basic assumptions is not satisfied, then the statement of the theorem becomes invalid.

Probability

What problem does this paper attempt to address?

The Invariance Principle for Some Class of Markov Chains

ZY Lin

DOI: https://doi.org/10.4213/tvp615

2000-01-01

Theory of Probability and Its Applications

Abstract:This work is devoted to the invariance principle for homogeneous Markov chains with a finite state space. The proof is based on a comparison of the asymptotic behavior of sums of random variables constructed by Markov chains with the same transition matrix and various initial distributions.
Recurrence for branching Markov chains

Sebastian Müller

DOI: https://doi.org/10.48550/arXiv.0710.4651

2008-11-12

Abstract:The question of recurrence and transience of branching Markov chains is more subtle than for ordinary Markov chains; they can be classified in transience, weak recurrence, and strong recurrence. We review criteria for transience and weak recurrence and give several new conditions for weak recurrence and strong recurrence. These conditions make a unified treatment of known and new examples possible and provide enough information to distinguish between weak and strong recurrence. This represents a step towards a general classification of branching Markov chains. In particular, we show that in \emph{homogeneous} cases weak recurrence and strong recurrence coincide. Furthermore, we discuss the generalization of positive and null recurrence to branching Markov chains and show that branching random walks on $\Z$ are either transient or positive recurrent.

Probability
The Key Renewal Theorem for a Transient Markov Chain

Dmitry Korshunov

DOI: https://doi.org/10.48550/arXiv.0711.2169

2007-11-14

Abstract:We consider a time-homogeneous Markov chain $X_n$, $n\ge0$, valued in ${\bf R}$. Suppose that this chain is transient, that is, $X_n$ generates a $\sigma$-finite renewal measure. We prove the key renewal theorem under condition that this chain has asymptotically homogeneous at infinity jumps and asymptotically positive drift.

Probability
On the Relation Between Reversibility and Monotonicity of Fluctuation Spectra for Discrete Time Finite State Markov Chains

Yong Chen,Min-Ping Qian,Han-Sheng Xie

DOI: https://doi.org/10.1016/j.spl.2008.01.095

IF: 0.718

2008-01-01

Statistics & Probability Letters

Abstract:For the irreducible aperiodic discrete time finite state Markov chain, (i) if it is reversible, we provide a necessary and sufficient condition for all fluctuation spectra to be monotonic on [0,π] and (ii) if it is irreversible, then there exist some nonmonotonic fluctuation spectra.
Asymptotic period of an aperiodic Markov chain

Erik A. van Doorn

DOI: https://doi.org/10.48550/arXiv.1712.10199

2018-11-28

Abstract:We introduce the concept of asymptotic period for an irreducible and aperiodic, discrete-time Markov chain X on a countable state space, and develop the theory leading to its formal definition. The asymptotic period of X equals one - its period - if X is recurrent, but may be larger than one if X is transient; X is asymptotically aperiodic if its asymptotic period equals one. Some sufficient conditions for asymptotic aperiodicity are presented. The asymptotic period of a birth-death process on the nonnegative integers is studied in detail and shown to be equal to 1, 2 or infinity. Criteria for the occurrence of each value in terms of the 1-step transition probabilities are established.

Probability
Characterization of cutoff for reversible Markov chains

Riddhipratim Basu,Jonathan Hermon,Yuval Peres

DOI: https://doi.org/10.1214/16-AOP1090

2015-02-18

Abstract:A sequence of Markov chains is said to exhibit (total variation) cutoff if the convergence to stationarity in total variation distance is abrupt. We consider reversible lazy chains. We prove a necessary and sufficient condition for the occurrence of the cutoff phenomena in terms of concentration of hitting time of "worst" (in some sense) sets of stationary measure at least $\alpha$, for some $\alpha \in (0,1)$. We also give general bounds on the total variation distance of a reversible chain at time $t$ in terms of the probability that some "worst" set of stationary measure at least $\alpha$ was not hit by time $t$. As an application of our techniques we show that a sequence of lazy Markov chains on finite trees exhibits a cutoff iff the ratio of their relaxation-times and their (lazy) mixing-times tends to 0.

Probability
Conditions for recurrence and transience for time-inhomogeneous random walks

Vyacheslav M. Abramov

2024-04-23

Abstract:The present paper extends the earlier results obtained by Abramov [`Conditions for recurrence and transience for time-inhomogeneous birth-and-death processes' \emph{Bull. Aust. Math. Soc.} \textbf{109} (2024), 393--402] for the case of time-inhomogeneous random walks, the increments of which take values in $\mathbb{R}$. By this, we give a full solution of the open problem formulated by Menshikov and Volkov [`Urn-related random walk with drift $\rho x^\alpha/t^\beta$' \emph{Electron. J. Probab.}, \textbf{13} (2008), paper No. 31, 944--960] that was partially solved in the aforementioned paper by Abramov.

Probability
Computing the Yaglom limit of Markov chains with a single exit state using their excursion measure

Elie Cerf

2024-06-10

Abstract:We prove in this article the existence of the Yaglom limit for Markov chains on discrete state spaces in the setting where the absorbing state is accessible from a single non-absorbing state. We use a representation of the trajectories of this process by its excursion away from death, that allows us to link the Yaglom limit with the large deviations behaviour of the inverse of its local time at the exit state, and to compute its minimal quasi-stationary distribution with its excursion measure.

Probability
Uniform syndeticity in multiple recurrence

ASGAR JAMNESHAN,MINGHAO PAN

DOI: https://doi.org/10.1017/etds.2024.40

2024-05-30

Ergodic Theory and Dynamical Systems

Abstract:The main theorem of this paper establishes a uniform syndeticity result concerning the multiple recurrence of measure-preserving actions on probability spaces. More precisely, for any integers and any , we prove the existence of and (dependent only on d, l, and ) such that the following holds: Consider a solvable group of derived length l, a probability space , and d pairwise commuting measure-preserving -actions on . Let E be a measurable set in X with . Then, K many (left) translates of cover . This result extends and refines uniformity results by Furstenberg and Katznelson. As a combinatorial application, we obtain the following uniformity result. For any integers and any , there are and (dependent only on d, l, and ) such that for all finite solvable groups G of derived length l and any subset with (where m is the uniform measure on G), we have that K-many (left) translates of cover G. The proof of our main result is a consequence of an ultralimit version of Austin's amenable ergodic Szeméredi theorem.

mathematics, applied
Explosion, implosion, and moments of passage times for continuous-time Markov chains: a semimartingale approach

Mikhail Menshikov,Dimitri Petritis

DOI: https://doi.org/10.1016/j.spa.2014.03.001

2012-06-14

Abstract:We establish general theorems quantifying the notion of recurrence --- through an estimation of the moments of passage times --- for irreducible continuous-time Markov chains on countably infinite state spaces. Sharp conditions of occurrence of the phenomenon of explosion are also obtained. A new phenomenon of implosion is introduced and sharp conditions for its occurrence are proven. The general results are illustrated by treating models having a difficult behaviour even in discrete time.

Probability,Mathematical Physics
Separating measurable recurrence from strong recurrence via rigidity sequences

John T. Griesmer

2024-06-17

Abstract:If $G$ is an abelian group, we say $S\subset G$ is a set of recurrence if for every probability measure preserving $G$-system $(X,\mu,T)$ and every $D\subset X$ having $\mu(D)>0$, there is a $g\in S$ such that $\mu(D\cap T^{g}D)>0$. We say $S$ is a set of strong recurrence if for every set $D$ having $\mu(D)>0$ there is a $c>0$ such that $\mu(D\cap T^{g}D)>c$ for infinitely many $g\in S$. We call $S$ measure expanding if for all $g\in G$, the translate $S+g$ is a set of recurrence. A rigidity sequence for $(X,\mu,T)$ is a sequence of elements $s_n\in G$ satisfying $\lim_{n\to\infty} \mu(D\triangle T^{s_n}D)=0$ for all measurable $D\subset X$. For all but countably many countable abelian groups $G$, we prove that if $S$ is measure expanding, there is a sequence of elements $s_n\in S$ such that $\{s_n:n\in \mathbb N\}$ is also measure expanding and every translate of $(s_n)$ is a rigidity sequence for some free weak mixing measure preserving $G$-system. The special case where $S=G$ proves a conjecture of Ackelsberg. As a consequence, we prove that for every countably infinite abelian group $G$ and every measure expanding set $S\subset G$ there is a subset $S'\subset S$ such that $S'$ is measure expanding and no translate of $S'$ is a set of strong recurrence.

Dynamical Systems
Beyond Decisiveness of Infinite Markov Chains

Benoît Barbot,Patricia Bouyer,Serge Haddad

2024-09-27

Abstract:Verification of infinite-state Markov chains is still a challenge despite several fruitful numerical or statistical approaches. For decisive Markov chains, there is a simple numerical algorithm that frames the reachability probability as accurately as required (however with an unknown complexity). On the other hand when applicable, statistical model checking is in most of the cases very efficient. Here we study the relation between these two approaches showing first that decisiveness is a necessary and sufficient condition for almost sure termination of statistical model checking. Afterwards we develop an approach with application to both methods that substitutes to a non decisive Markov chain a decisive Markov chain with the same reachability probability. This approach combines two key ingredients: abstraction and importance sampling (a technique that was formerly used for efficiency). We develop this approach on a generic formalism called layered Markov chain (LMC). Afterwards we perform an empirical study on probabilistic pushdown automata (an instance of LMC) to understand the complexity factors of the statistical and numerical algorithms. To the best of our knowledge, this prototype is the first implementation of the deterministic algorithm for decisive Markov chains and required us to solve several qualitative and numerical issues.

Logic in Computer Science,Formal Languages and Automata Theory
Markov chains: Structure and applications

R.L. Tweedie

DOI: https://doi.org/10.1016/s0169-7161(01)19025-5

2001-01-01

Handbook of Statistics

Abstract:Publisher The classical theory of Markov chains originally dealt only with chains on finite or countable spaces. In order to establish the fundamental aspects of Markov chain theory on more general spaces, the chapter presents an overview of results on countable spaces, including some recent and less familiar aspects even on these spaces. Although many real systems can be described in such a framework, many other applications require continuous or even more general spaces, and there has been a flourishing of work on non-countable spaces. This chapter integrates the general state space results with topological conditions, which guarantee that the obvious (but not always true) probabilistic properties are preserved when the space has topological features.
Conditions for recurrence and transience for one family of random walks

Vyacheslav M. Abramov

2023-07-23

Abstract:A parametric family of two-dimensional random walks $\mathbf{S}_t(a)$ $=\big(S_t^{(1)}(a),$ $S_t^{(2)}(a)\big)$ in the main quarter plane is studied. The components $S_t^{(1)}(a)$ and $S_t^{(2)}(a)$ are assumed to be correlated in the way that is defined exactly in the paper. We derive the conditions on $a$, under which a random walk $\mathbf{S}_t(a)$ is recurrent.

Probability,Combinatorics
A transient Markov chain with finitely many cutpoints

Nicholas James,Russell Lyons,Yuval Peres

DOI: https://doi.org/10.1214/193940307000000365

2008-05-19

Abstract:We give an example of a transient reversible Markov chain that almost surely has only a finite number of cutpoints. We explain how this is relevant to a conjecture of Diaconis and Freedman and a question of Kaimanovich. We also answer Kaimanovich's question when the Markov chain is a nearest-neighbor random walk on a tree.

Probability
On regularity of functions of Markov chains

Steven Berghout,Evgeny Verbitskiy

DOI: https://doi.org/10.48550/arXiv.2101.00608

2021-01-03

Abstract:We consider processes which are functions of finite-state Markov chains. It is well known that such processes are rarely Markov. However, such processes are often regular in the following sense: the distant past values of the process have diminishing influence on the distribution of the present value. In the present paper, we present novel sufficient conditions for regularity of functions of Markov chains.

Probability,Dynamical Systems
Countable Markov shifts with exponential mixing

Mike Todd,Boyuan Zhao

2024-03-04

Abstract:We give a set of equivalent conditions for a potential on a Countable Markov Shift to have strong positive recurrence, which is also equivalent to having exponential decay of correlations. A key ingredient of our proofs is quantifying how the shift behaves at its boundary.

Dynamical Systems
Cutoff for mixtures of permuted Markov chains: reversible case

Bastien Dubail

2024-02-08

Abstract:We investigate the mixing properties of a model of reversible Markov chains in random environment, which notably contains the simple random walk on the superposition of a deterministic graph and a second graph whose vertex set has been permuted uniformly at random. It generalizes in particular a result of Hermon, Sly and Sousi, who proved the cutoff phenomenon at entropic time for the simple random walk on a graph with an added uniform matching. Under mild assumptions on the base Markov chains, we prove that with high probability the resulting chain exhibits the cutoff phenomenon at entropic time log n/h, h being some constant related to the entropy of the chain. We note that the results presented here are the consequence of a work conducted for a more general model that does not assume reversibility, which will be the object of a companion paper. Thus, most of our proofs do not actually require reversibility, which constitutes an important technical contribution. Finally, our argument relies on a novel concentration result for "low-degree" functions on the symmetric group, established specifically for our purpose but which could be of independent interest.

Probability
Quasi-stationary distributions in reducible state spaces

Nicolas Champagnat,Denis Villemonais

2024-07-09

Abstract:We study quasi-stationary distributions and quasi-limiting behavior of Markov chains in general reducible state spaces with absorption. We propose a set of assumptions dealing with particular situations where the state space can be decomposed into three subsets between which communication is only possible in a single direction. These assumptions allow us to characterize the exponential order of magnitude and the exact polynomial correction, called polynomial convergence parameter, for the leading order term of the semigroup for large time. They also provide explicit convergence speeds to this leading order term. We apply these results to general Markov chains with finitely or denumerably many communication classes using a specific induction over the communication classes of the chain. We are able to explicitely characterize the polynomial convergence parameter, to determine the complete set of quasistationary distributions and to provide explicit estimates for the speed of convergence to quasi-limiting distributions in the case of finitely many communication classes. We conclude with an application of these results to the case of denumerable state spaces, where we are able to prove that, in general, there is existence of a quasi-stationary distribution without assuming irreducibility before absorption. This actually holds true assuming only aperiodicity, the existence of a Lyapunov function and the existence of a point in the state space from which the return time is finite with positive probability.

Probability

A new criterion for recurrence of Markov chains with an infinitely countable set of states

The Invariance Principle for Some Class of Markov Chains

Recurrence for branching Markov chains

The Key Renewal Theorem for a Transient Markov Chain

On the Relation Between Reversibility and Monotonicity of Fluctuation Spectra for Discrete Time Finite State Markov Chains

Asymptotic period of an aperiodic Markov chain

Characterization of cutoff for reversible Markov chains

Conditions for recurrence and transience for time-inhomogeneous random walks

Computing the Yaglom limit of Markov chains with a single exit state using their excursion measure

Uniform syndeticity in multiple recurrence

Explosion, implosion, and moments of passage times for continuous-time Markov chains: a semimartingale approach

Separating measurable recurrence from strong recurrence via rigidity sequences

Beyond Decisiveness of Infinite Markov Chains

Markov chains: Structure and applications

Conditions for recurrence and transience for one family of random walks

A transient Markov chain with finitely many cutpoints

On regularity of functions of Markov chains

Countable Markov shifts with exponential mixing

Cutoff for mixtures of permuted Markov chains: reversible case

Quasi-stationary distributions in reducible state spaces