A Markov Chain Approach to the Problem of Runs and Patterns

Zhiying Liang

1993-01-01

Abstract:A new approach. Converts the problem of runs and patterns into a problem of Markov chain with discrete state space.

What problem does this paper attempt to address?

Advanced method and application on Markov state transfer chain

Xi Li,Xiaoning Zhu,Guoqiang Cai,Yuan Sun

2008-01-01

Abstract:This paper advanced the solution of Markov state transfer chain to analyze the system reliability. According to the conversion of mutual relations, the paper put forward a method to calculate the life expectancy of top events in Markov quantitative analysis. Afterwards the paper presented the failure mode, failure rate and life expectancy of the top events. Meanwhile, the paper presented the method to calculate probability of important bottom events to top events. Finally the research is verified in the analysis of a city rail traffic automatic train super-vision system.
The Invariance Principle for Some Class of Markov Chains

ZY Lin

DOI: https://doi.org/10.4213/tvp615

2000-01-01

Theory of Probability and Its Applications

Abstract:This work is devoted to the invariance principle for homogeneous Markov chains with a finite state space. The proof is based on a comparison of the asymptotic behavior of sums of random variables constructed by Markov chains with the same transition matrix and various initial distributions.
A unified Markov chain approach for computing the run length distribution in control charts with simple or compound rules

James C. Fu,Galit Shmueli,Y.M. Chang

DOI: https://doi.org/10.1016/j.spl.2003.10.004

IF: 0.718

2003-01-01

Statistics & Probability Letters

Abstract:We introduce a general theoretical framework, based on the Markov chain imbedding approach, that leads to the run-length distribution for a multitude of control charts that are based on a simple rule or on a compound set of rules.
Markov chains: Structure and applications

R.L. Tweedie

DOI: https://doi.org/10.1016/s0169-7161(01)19025-5

2001-01-01

Handbook of Statistics

Abstract:Publisher The classical theory of Markov chains originally dealt only with chains on finite or countable spaces. In order to establish the fundamental aspects of Markov chain theory on more general spaces, the chapter presents an overview of results on countable spaces, including some recent and less familiar aspects even on these spaces. Although many real systems can be described in such a framework, many other applications require continuous or even more general spaces, and there has been a flourishing of work on non-countable spaces. This chapter integrates the general state space results with topological conditions, which guarantee that the obvious (but not always true) probabilistic properties are preserved when the space has topological features.
Waiting times and stopping probabilities for patterns in Markov chains

Min-zhi Zhao,Dong Xu,Hui-zeng Zhang

DOI: https://doi.org/10.1007/s11766-018-3522-z

2018-03-01

Abstract:Suppose that C is a finite collection of patterns. Observe a Markov chain until one of the patterns in C occurs as a run. This time is denoted by τ. In this paper, we aim to give an easy way to calculate the mean waiting time E(τ ) and the stopping probabilities P(τ = τA) with A ∈ C, where τA is the waiting time until the pattern A appears as a run.
An effective introduction to the Markov Chain Monte Carlo method

Wenlong Wang

DOI: https://doi.org/10.48550/arXiv.2204.10145

2022-04-21

Computational Physics

Abstract:We present an intuitive, conceptual, but semi-rigorous introduction to the celebrated Markov Chain Monte Carlo method using a simple model of population dynamics as our motivation and focusing on a few elementary distributions. Conceptually, the population flow between cities closely resembles the random walk of a single walker in a state space. We start from two states, then three states, and finally the setup is fully generalized to many states of both discrete and continuous distributions. Despite the mathematical simplicity, the setup remarkably includes all the essential concepts of Markov Chain Monte Carlo without loss of generality, e.g., ergodicity, global balance and detailed balance, proposal or selection probability, acceptance probability, up to the underlying stochastic matrix, and error analysis. Our teaching experience suggests that most senior undergraduate students in physics can closely follow these materials without much difficulty.
Multiple pattern matching: A Markov chain approach

Manuel Lladser,M. D. Betterton,Rob Knight

DOI: https://doi.org/10.48550/arXiv.0704.3221

2007-04-25

Abstract:RNA motifs typically consist of short, modular patterns that include base pairs formed within and between modules. Estimating the abundance of these patterns is of fundamental importance for assessing the statistical significance of matches in genomewide searches, and for predicting whether a given function has evolved many times in different species or arose from a single common ancestor. In this manuscript, we review in an integrated and self-contained manner some basic concepts of automata theory, generating functions and transfer matrix methods that are relevant to pattern analysis in biological sequences. We formalize, in a general framework, the concept of Markov chain embedding to analyze patterns in random strings produced by a memoryless source. This conceptualization, together with the capability of automata to recognize complicated patterns, allows a systematic analysis of problems related to the occurrence and frequency of patterns in random strings. The applications we present focus on the concept of synchronization of automata, as well as automata used to search for a finite number of keywords (including sets of patterns generated according to base pairing rules) in a general text.

Probability,Combinatorics,Statistics Theory,Genomics,Quantitative Methods
An introduction to the Markov chain Monte Carlo method

Wenlong Wang

DOI: https://doi.org/10.1119/5.0122488

IF: 0.835

2022-12-02

American Journal of Physics

Abstract:We present an intuitive, conceptual, and semi-rigorous introduction to the Markov Chain Monte Carlo method using a simple model of population dynamics and focusing on a few elementary distributions. We start from two states, then three states, and finally generalize to many states with both discrete and continuous distributions. Despite the mathematical simplicity, our examples include the essential concepts of the Markov Chain Monte Carlo method, including ergodicity, global balance and detailed balance, proposal or selection probability, acceptance probability, the underlying stochastic matrix, and error analysis. Our experience suggests that most senior undergraduate students in physics can follow these materials without much difficulty.
Markov Chain Monte Carlo: The Basics

Adrian Barbu,Song-Chun Zhu

DOI: https://doi.org/10.1007/978-981-13-2971-5_3

2020-01-01

Abstract:Imagine you enter a big national park (in the poem it is a mountain), your path is essentially a Markov chain in a bounded space. The frequency that you stay at an attraction spot is proportional to its popularity.
Synthesizing Markov Chain With Reversible Unimolecular Reactions

Ziyuan Shen,Lulu Ge,Wei Wei,Jing Zhao,Xiaohu You,Chuan Zhang

DOI: https://doi.org/10.1109/WCSP.2017.8171000

2017-01-01

Abstract:It is prevalently known that Markov chains have been successfully applied to many fields such as digital communications, queuing theory, and finance. However, when the system becomes massive and complex, the computational load will become too heavy to be handled by conventional approaches. To address this issue, molecular computation, which is inherently parallel, has been considered in this paper to synthesize Markov chain. Here, a succinct and systematic approach based on reversible unimolecular reactions is proposed for any time-homogeneous Markov chain, no matter it is discrete or continuous. For the chemical reaction networks (CRNs), molecular concentrations at time t reflect the probability distribution of the continuous-time Markov chain at time t. The final concentrations indicate the steady state of the Markov chain. Numerical results based on deterministic mass action kinetics have demonstrated the robustness and accuracy of this method. It is worth noting that an already mathematically-proven conclusion, which states that nearly an arbitrary set of uni-or bimolecular reactions can be implemented by DNA strand displacement reactions, ensures the meaningfulness of our work.
Markov Chains On Continuous State Space

Quan-Lin Li

DOI: https://doi.org/10.1007/978-3-642-11492-2_5

2010-01-01

Abstract:In this chapter, we discuss Markov chains on continuous state space. We first analyze a discrete-time Markov chain on continuous state space, and then discuss a discrete-time Markov chain on a bivariate state space. Applying the censoring technique, we provide expression for the RG-factorizations, which are used to derive the stationary probability of the Markov chain. Further, we consider a continuous-time Markov chain on continuous state space. Specifically, we deal with a continuous-time level-dependent QBD process with continuous phase variable, and provide orthonormal representations for the R-, U- and G-measures, which lead to the matrix-structured computation of the stationary probability. As an application, we introduce continuous-phase type (CPH) distribution and continuous-phase Markovian arrival process (CMAP), and then analyze a CMAP/CPH/1 queue. Finally, we study a piecewise deterministic Markov process, which is applied to deal with more general queues such as the GI/G/c queue.
A Tutorial on the Spectral Theory of Markov Chains

Eddie Seabrook,Laurenz Wiskott

DOI: https://doi.org/10.1162/neco_a_01611

2022-08-19

Abstract:Markov chains are a class of probabilistic models that have achieved widespread application in the quantitative sciences. This is in part due to their versatility, but is compounded by the ease with which they can be probed analytically. This tutorial provides an in-depth introduction to Markov chains, and explores their connection to graphs and random walks. We utilize tools from linear algebra and graph theory to describe the transition matrices of different types of Markov chains, with a particular focus on exploring properties of the eigenvalues and eigenvectors corresponding to these matrices. The results presented are relevant to a number of methods in machine learning and data mining, which we describe at various stages. Rather than being a novel academic study in its own right, this text presents a collection of known results, together with some new concepts. Moreover, the tutorial focuses on offering intuition to readers rather than formal understanding, and only assumes basic exposure to concepts from linear algebra and probability theory. It is therefore accessible to students and researchers from a wide variety of disciplines.

Machine Learning
The Markov-Chain Polytope with Applications

Mordecai J. Golin,Albert John Lalim Patupat

2024-08-17

Abstract:This paper addresses the problem of finding a minimum-cost $m$-state Markov chain $(S_0,\ldots,S_{m-1})$ in a large set of chains. The chains studied have a reward associated with each state. The cost of a chain is its "gain", i.e., its average reward under its stationary distribution. Specifically, for each $k=0,\ldots,m-1$ there is a known set ${\mathbb S}_k$ of type-$k$ states. A permissible Markov chain contains exactly one state of each type; the problem is to find a minimum-cost permissible chain. The original motivation was to find a cheapest binary AIFV-$m$ lossless code on a source alphabet of size $n$. Such a code is an $m$-tuple of trees, in which each tree can be viewed as a Markov Chain state. This formulation was then used to address other problems in lossless compression. The known solution techniques for finding minimum-cost Markov chains were iterative and ran in exponential time. This paper shows how to map every possible type-$k$ state into a type-$k$ hyperplane and then define a "Markov Chain Polytope" as the lower envelope of all such hyperplanes. Finding a minimum-cost Markov chain can then be shown to be equivalent to finding a "highest" point on this polytope. The local optimization procedures used in the previous iterative algorithms are shown to be separation oracles for this polytope. Since these were often polynomial time, an application of the Ellipsoid method immediately leads to polynomial time algorithms for these problems.

Information Theory
A Study in Markov Chains, Loop-Erased Random Walk and Loop Soups

Zhuohan Gu

2024-11-15

Abstract:In this paper, we make a few random explorations that relate directly to the items mentioned in the title. We define transient chains and recurrent chains with "killing", the Green's function, the Laplacian operator, and harmonic functions. We then introduce the loop-erased random walk (LERW) and its relationship with the uniform spanning tree (UST). We finish by introducing loop measures and soups and defining the "growing loop" model.

Probability
Fluctuation Theory of Continuous-Time, Skip-Free Downward Markov Chains with Applications to Branching Processes with Immigration

Ronnie Loeffen,Pierre Patie,Jian Wang

DOI: https://doi.org/10.1287/moor.2022.0246

IF: 2.215

2024-02-14

Mathematics of Operations Research

Abstract:We develop a comprehensive methodology for the fluctuation theory of continuous-time, skip-free Markov chains, extending and improving the recent work of Choi and Patie for discrete-time, skip-free Markov chains. As a significant application, we use it to derive a full set of fluctuation identities regarding exiting a finite or infinite interval for Markov branching processes with immigration, thereby uncovering many new results for this classic family of continuous-time Markov chains. The theory also allows us to recover in a simple manner fluctuation identities for skip-free downward compound Poisson processes.

mathematics, applied,operations research & management science
Identifying Markov chain models from time-to-event data: an algebraic approach

Ovidiu Radulescu,Dima Grigoriev,Matthias Seiss,Maria Douaihy,Mounia Lagha,Edouard Bertrand

2023-11-07

Abstract:In many fields, including biology, medicine, physics, chemistry, economy and actuarial science, data can be represented as the time-to-event of a finite state Markov chain model. The distribution of time intervals between successive recorded events is known as a phase-type distribution. We demonstrate that in cases where the eigenvalues of the Markov chain transition matrix are distinct (non-degenerate), the phase-type distribution is multi-exponential. We then pose and solve an inverse problem: given knowledge of the phase-type distribution, can we determine the transition rate parameters of the underlying Markov chain? To tackle this challenge, we initially convert the inverse problem into a computer algebraic task, involving the solution of a system of polynomial equations. These equations are symmetrized with respect to the parameters of the phase-type distribution. For a specific subset of Markov models that we refer to as "solvable," the inverse problem yields a unique solution, up to transformations by finite symmetries. We outline a recursive approach to compute these solutions for specific families of models, regardless of their number of states. Additionally, we use the Thomas decomposition technique to calculate solutions for models possessing 2, 3, or 4 states. Interestingly, we show that models having the same number of states but different transition graphs can yield identical phase-distributions. In such "Rashomon effect" scenarios, time-to-event data permits the formulation of multiple models and interpretations, all of which are consistent with the observed experimental data. To differentiate among these models, we propose additional distinguishing properties beyond just the time until the next event. In order to prove its applicability, we discuss how this method can be used to infer models of transcription regulation from transcriptional bursting data.

Dynamical Systems,Quantitative Methods
Some properties of Markov chains on the free group $\mathbb F_2$

Antoine Goldsborough,Stefanie Zbinden

2023-09-26

Abstract:Random walks cannot, in general, be pushed forward by quasi-isometries. Tame Markov chains were introduced as a `quasi-isometry invariant' are a generalization of random walks. In this paper, we construct several examples of tame Markov chains on the free group exhibiting `exotic' behaviour; one, where the drift is not well defined and one where the drift is well defined but the Central Limit Theorem does not hold. We show that this is not a failure of the notion of tame Markov chain, but rather that any quasi-isometry invariant theory that generalizes random walks will include examples without well-defined drift.

Group Theory,Probability
Research on Pedestrian Escape Route Without Visibility Based on Markov Chain Model

Xiaoping Zheng,Lijuan Du,Hongjiang Wang

DOI: https://doi.org/10.1007/s11705-009-0272-8

2009-01-01

Frontiers of Chemical Engineering in China

Abstract:It is a pioneering work to use a Markov chain model to study the pedestrian escape route without visibility. In this paper, based on the Markov chain probability transition matrix, the algorithms with random numbers and the spatial-grid, an escape route in a limited invisible space is obtained. Six pace states (standing, crawling, walking, leaping, jogging, and running) are applied to describe the characteristics of pedestrian behaviors. Besides, eight main direction changes are used to describe the transition characteristic of a pedestrian. At the same time, this paper analyzes the escape route from two views, i.e., pedestrian pace states and directions. The research results show that the Markov chain model is more realistic as a means of studying pedestrian escape routes.
From Random Walks to Random Leaps: Generalizing Classic Markov Chains for Big Data Applications

Bala Rajaratnam,Narut Sereewattanawoot,Doug Sparks,Meng-Hsuan Wu

DOI: https://doi.org/10.48550/arXiv.1708.03116

2017-08-10

Probability

Abstract:Simple random walks are a basic staple of the foundation of probability theory and form the building block of many useful and complex stochastic processes. In this paper we study a natural generalization of the random walk to a process in which the allowed step sizes take values in the set $\{\pm1,\pm2,\ldots,\pm k\}$, a process we call a random leap. The need to analyze such models arises naturally in modern-day data science and so-called "big data" applications. We provide closed-form expressions for quantities associated with first passage times and absorption events of random leaps. These expressions are formulated in terms of the roots of the characteristic polynomial of a certain recurrence relation associated with the transition probabilities. Our analysis shows that the expressions for absorption probabilities for the classical simple random walk are a special case of a universal result that is very elegant. We also consider an important variant of a random leap: the reflecting random leap. We demonstrate that the reflecting random leap exhibits more interesting behavior in regard to the existence of a stationary distribution and properties thereof. Questions relating to recurrence/transience are also addressed, as well as an application of the random leap.
Markov chains and mappings of distributions on compact spaces

David J. Aldous,Shi Feng

2024-04-01

Abstract:Consider a compact metric space $S$ and a pair $(j,k)$ with $k \ge 2$ and $1 \le j \le k$. For any probability distribution $\theta \in P(S)$, define a Markov chain on $S$ by: from state $s$, take $k$ i.i.d. ($\theta$) samples, and jump to the $j$'th closest. Such a chain converges in distribution to a unique stationary distribution, say $\pi_{j,k}(\theta)$. So this defines a mapping $\pi_{j,k}: P(S) \to P(S)$. What happens when we iterate this mapping? In particular, what are the fixed points of this mapping? We present a few rigorous results, to complement our extensive simulation study elsewhere.

Probability,Metric Geometry

A Markov Chain Approach to the Problem of Runs and Patterns

Advanced method and application on Markov state transfer chain

The Invariance Principle for Some Class of Markov Chains

A unified Markov chain approach for computing the run length distribution in control charts with simple or compound rules

Markov chains: Structure and applications

Waiting times and stopping probabilities for patterns in Markov chains

An effective introduction to the Markov Chain Monte Carlo method

Multiple pattern matching: A Markov chain approach

An introduction to the Markov chain Monte Carlo method

Markov Chain Monte Carlo: The Basics

Synthesizing Markov Chain With Reversible Unimolecular Reactions

Markov Chains On Continuous State Space

A Tutorial on the Spectral Theory of Markov Chains

The Markov-Chain Polytope with Applications

A Study in Markov Chains, Loop-Erased Random Walk and Loop Soups

Fluctuation Theory of Continuous-Time, Skip-Free Downward Markov Chains with Applications to Branching Processes with Immigration

Identifying Markov chain models from time-to-event data: an algebraic approach

Some properties of Markov chains on the free group $\mathbb F_2$

Research on Pedestrian Escape Route Without Visibility Based on Markov Chain Model

From Random Walks to Random Leaps: Generalizing Classic Markov Chains for Big Data Applications

Markov chains and mappings of distributions on compact spaces