Collective marks and first passage times

Yiping ZHANG,Myron HLYNKA,Percy H. BRILL

DOI: https://doi.org/10.48550/arXiv.1908.04370

2019-08-13

Abstract:Probability generating functions for first passage times of Markov chains are found using the method of collective marks. A system of equations is found which can be used to obtain moments of the first passage times.

Probability

What problem does this paper attempt to address?

Higher Order Ergodic Markov Chains and First Passage Times

Lixing Han,Kelun Wang,Jianhong Xu

DOI: https://doi.org/10.1080/03081087.2021.1968782

2022-01-01

Abstract:We study the relationship between ergodicity and irreducibility of transition probability tensors. We then introduce the notions of first passage times and mean first passage times, both in the tensor form, for higher order Markov chains and prove some properties of these quantities. Numerical examples are also given to illustrate our results.
First Passage Times for Markov Renewal Processes and Applications

Guanghui Xu,Xueming Yuan,Quanlin Li

DOI: https://doi.org/10.1007/bf02880061

2000-01-01

Science in China Series A Mathematics

Abstract:This paper proposes a uniformly convergent algorithm for the joint transform of the first passage time and the first passage number of steps for general Markov renewal processes with any initial state probability vector. The uniformly convergent algorithm with arbitrarily prescribed error can be efficiently applied to compute busy periods, busy cycles, waiting times, sojourn times, and relevant indices of various generic queueing systems and queueing networks. This paper also conducts a numerical experiment to implement the proposed algorithm.
Waiting times and stopping probabilities for patterns in Markov chains

Min-zhi Zhao,Dong Xu,Hui-zeng Zhang

DOI: https://doi.org/10.1007/s11766-018-3522-z

2018-03-01

Abstract:Suppose that C is a finite collection of patterns. Observe a Markov chain until one of the patterns in C occurs as a run. This time is denoted by τ. In this paper, we aim to give an easy way to calculate the mean waiting time E(τ ) and the stopping probabilities P(τ = τA) with A ∈ C, where τA is the waiting time until the pattern A appears as a run.
The First-Passage Times of Phase Semi-Markov Processes

Xuan Zhang,Zhenting Hou

DOI: https://doi.org/10.1016/j.spl.2011.08.021

IF: 0.718

2012-01-01

Statistics & Probability Letters

Abstract:In this paper, we consider a class of semi-Markov processes, known as phase semi-Markov processes, which can be considered as an extension of Markov processes, but whose times between transitions are phase-type random variables. Based on the theory of generalized inverses, we derive expressions for the moments of the first-passage time distributions, generalizing the results obtained by Kemeny and Snell (1960) for Markov chains.
First-passage times in renewal and nonrenewal systems

Krzysztof Ptaszynski

DOI: https://doi.org/10.1103/PhysRevE.97.012127

2018-01-11

Abstract:Fluctuations in stochastic systems are usually characterized by the full counting statistics, which analyzes the distribution of the number of events taking place in the fixed time interval. In an alternative approach, the distribution of the first-passage times, i.e. the time delays after which the counting variable reaches a certain threshold value, is studied. This paper presents the approach to calculate the first-passage time distribution in systems in which the analyzed current is associated with an arbitrary set of transitions within the Markovian network. Using this approach it is shown that when the subsequent first-passage times are uncorrelated there exist strict relations between the cumulants of the full counting statistics and the first-passage time distribution. On the other hand, when the correlations of the first-passage times are present, their distribution may provide additional information about the internal dynamics of the system in comparison to the full counting statistics; for example, it may reveal the switching between different dynamical states of the system. Additionally, I show that breaking of the fluctuation theorem for first-passage times may reveal the multicyclic nature of the Markovian network.

Statistical Mechanics
Exact analytic expressions for discrete first-passage time probability distributions in Markov networks

Jaroslav Albert

2024-03-21

Abstract:The first-passage time (FPT) is the time it takes a system variable to cross a given boundary for the first time. In the context of Markov networks, the FPT is the time a random walker takes to reach a particular node (target) by hopping from one node to another. If the walker pauses at each node for a period of time drawn from a continuous distribution, the FPT will be a continuous variable; if the pauses last exactly one unit of time, the FPT will be discrete and equal to the number of hops. We derive an exact analytical expression for the discrete first-passage time (DFPT) in Markov networks. Our approach is as follows: first, we divide each edge (connection between two nodes) of the network into $h$ unidirectional edges connecting a cascade of $h$ fictitious nodes and compute the continuous FPT (CFPT). Second, we set the transition rates along the edges to $h$, and show that as $h\to\infty$, the distribution of travel times between any two nodes of the original network approaches a delta function centered at 1, which is equivalent to pauses lasting 1 unit of time. Using this approach, we also compute the joint-probability distributions for the DFPT, the target node, and the node from which the target node was reached. A comparison with simulation confirms the validity of our approach.

Molecular Networks
New First Passage Times and Their Distributions

Lirong Cui,Jianhui Chen,Hongda Gao,Jingyuan Shen

DOI: https://doi.org/10.1109/smrlo.2016.14

2016-01-01

Abstract:The First Passage Time (FPT) is a very important index in the real world, which is ubiquitous in physics, chemistry, and biology, with numerous examples of applications, especially in reliability field, the lifetime is a kind of FPTs. A huge literature has been given on the FPTs, some of them have been appeared in Nature, Science and other famous journals. However, the current FPT cannot cover the increasing needs in scientific research and engineering. Therefore, some new FPTs are needed from a point-view of theory and applications. In this note, two new FPT are introduced for a given finite state and continuous time Markov process. After giving the definitions of FPTs, the detailed contents such as distributions of FPTs and the distributions of the number of sojourn times until the realizations of the FPTs are presented. The Laplace transform technique is used in the derivation of formulae. Finally, the future researches on FPTs shall be discussed.
A new method for first-passage time probability density function of structures

Chunlin Zheng,Jie Chen,Jun He,Xinjian Kou

DOI: https://doi.org/10.3969/j.issn.1000-1301.2006.04.009

2006-01-01

Abstract:Based on the integral equation method, which is presented by Madsen and so on, the conditional upcrossing rate, the joint upcrossing rate and the first-passage time probability density are obtained by use of single-point FORM method for first-passage of structures. The present method is suitable to analyze the first-passage failure of brittle structures due to its simple calculation and accurate result.
The First Passage Time Problem for Mixed-Exponential Jump Processes with Applications in Insurance and Finance

Chuancun Yin,Yuzhen Wen,Zhaojun Zong,Ying Shen

DOI: https://doi.org/10.1155/2014/571724

2014-01-01

Abstract and Applied Analysis

Abstract:This paper studies the first passage times to constant boundaries for mixed-exponential jump diffusion processes. Explicit solutions of the Laplace transforms of the distribution of the first passage times, the joint distribution of the first passage times and undershoot (overshoot) are obtained. As applications, we present explicit expression of the Gerber-Shiu functions for surplus processes with two-sided jumps, present the analytical solutions for popular path-dependent options such as lookback and barrier options in terms of Laplace transforms, and give a closed-form expression on the price of the zero-coupon bond under a structural credit risk model with jumps.
Higher-order statistics-based estimation of moments of first passage time for Markov chains in its reverse problem

Shi Yu,Zhang Xianda

DOI: https://doi.org/10.1109/ICOSP.1998.770244

1998-01-01

Abstract:This paper presents a statistical approach for estimating the moments by using higher-order cumulants when the additive noise is Gaussian colored noise. It is shown that the kth-order moments of the noise samples are only related to the variance of the noise and that the estimators have the properties of unbiasedness and congruence. A numerical simulation is performed for constructing a Markov chain of continuous time parameters. The results show the validity of this algorithm
Circulations of Markov-Chains with Continuous-Time and the Probability Interpretation of Some Determinants

MP QIAN,M QIAN,C QIAN

1984-01-01

Abstract:In this paper, the relation between the irreversibility and circulation of a Markov chainwith continuous time parameter is considered. By the calculation of determinants, it showshow the probability approach in[1] is related to T. Hill's results.
First Passage Probabilities of One-Dimensional Diffusion Processes

Huijie Ji,Jinghai Shao

DOI: https://doi.org/10.1007/s11464-015-0459-x

2015-01-01

Abstract:This work is devoted to calculating the first passage probabilities of one-dimensional diffusion processes. For a one-dimensional diffusion process, we construct a sequence of Markov chains so that their absorption probabilities approximate the first passage probability of the given diffusion process. This method is especially useful when dealing with time-dependent boundaries.
On the Longest Run and the Waiting Time for the First Run in a Continuous Time Multi-State Markov Chain

Eutichia Vaggelatou

DOI: https://doi.org/10.1007/s11009-024-10127-5

2024-11-28

Methodology And Computing In Applied Probability

Abstract:In this paper, a marked point process with types of marks ( r types of successes and a failure F ), , that appear in continuous time according to a continuous-time Markov chain is considered. By constructing an appropriate embedded process using Markov chain embedding technique in continuous time, the exact distribution and its Laplace transform for the waiting time T until the first appearance of an -run of length , for (whichever comes first), are provided. The exact distribution of the length of the longest run of successes in the time interval [0, t ] is also derived. Further, the asymptotic distributions of T and are obtained under general assumptions. Finally, numerical examples and applications in reliability theory, quality control and hypothesis testing are presented.

statistics & probability
The Distribution of Mixing Times in Markov Chains

Jeffrey J. Hunter

DOI: https://doi.org/10.1142/S0217595912500455

2011-11-01

Abstract:The distribution of the "mixing time" or the "time to stationarity" in a discrete time irreducible Markov chain, starting in state i, can be defined as the number of trials to reach a state sampled from the stationary distribution of the Markov chain. Expressions for the probability generating function, and hence the probability distribution of the mixing time starting in state i are derived and special cases explored. This extends the results of the author regarding the expected time to mixing [J.J. Hunter, Mixing times with applications to perturbed Markov chains, Linear Algebra Appl. 417 (2006) 108-123], and the variance of the times to mixing, [J.J. Hunter, Variances of first passage times in a Markov chain with applications to mixing times, Linear Algebra Appl. 429 (2008) 1135-1162]. Some new results for the distribution of recurrence and first passage times in three-state Markov chain are also presented.

Probability
Bounds on Fluctuations of First Passage Times for Counting Observables in Classical and Quantum Markov Processes

George Bakewell-Smith,Federico Girotti,Mădălin Guţă,Juan P. Garrahan

2024-05-16

Abstract:We study the statistics of first passage times (FPTs) of trajectory observables in both classical and quantum Markov processes. We consider specifically the FPTs of counting observables, that is, the times to reach a certain threshold of a trajectory quantity which takes values in the positive integers and is non-decreasing in time. For classical continuous-time Markov chains we rigorously prove: (i) a large deviation principle (LDP) for FPTs, whose corollary is a strong law of large numbers; (ii) a concentration inequality for the FPT of the dynamical activity, which provides an upper bound to the probability of its fluctuations to all orders; and (iii) an upper bound to the probability of the tails for the FPT of an arbitrary counting observable. For quantum Markov processes we rigorously prove: (iv) the quantum version of the LDP, and subsequent strong law of large numbers, for the FPTs of generic counts of quantum jumps; (v) a concentration bound for the the FPT of total number of quantum jumps, which provides an upper bound to the probability of its fluctuations to all orders, together with a similar bound for the sub-class of quantum reset processes which requires less strict irreducibility conditions; and (vi) a tail bound for the FPT of arbitrary counts. Our results allow to extend to FPTs the so-called "inverse thermodynamic uncertainty relations" that upper bound the size of fluctuations in time-integrated quantities. We illustrate our results with simple examples.

Statistical Mechanics,Quantum Physics
Semi-Markov Processes in Open Quantum Systems. III. Large Deviations of First Passage Time Statistics

Fei Liu,Shihao Xia,Shanhe Su

2024-10-08

Abstract:In a specific class of open quantum systems with finite and fixed numbers of collapsed quantum states, the semi-Markov process method is used to calculate the large deviations of the first passage time statistics. The core formula is an equation of poles, which is also applied in determining the scaled generating functions (SCGFs) of the counting statistics. For simple counting variables, the SCGFs of the first passage time statistics are derived by finding the largest modulus of the roots of this equation with respect to the $z$-transform parameter and then calculating its logarithm. The procedure is analogous to that of solving for the SCGFs of the counting statistics. However, for current-like variables, the method generally fails unless the equation of pole is simplified to a quadratic form. The fundamental reason for this lies in the nonuniqueness between the roots and the region of convergence for the joint transform. We illustrate these results via a resonantly driven two-level quantum system, where for several counting variables the solutions to the SCGFs of the first passage time are analytically obtained. Furthermore, we apply these functions to investigate quantum violations of the classical kinetic and thermodynamic uncertainty relations.

Statistical Mechanics,Quantum Physics
Estimation Method and Its Properties of Moments of First Passage Time for Markov Chains in Reverse Problems

Yuanlie Lin

1998-01-01

Abstract:This paper presents a statistical approach for estimating the moments of the first passage time for Markov chains under a background of independent additive Gaussian colored noise in the reverse problem for Markov chains. By using the comulents, it is obtained that the k order moment of the noisy samples is only related to the variance of the noise, and is independent to any other order moments of the noise. It can also be proved that the estimators obtained from this algorithm have the properties of unbiasedness and congruence. A numerical simulation is performed for constructing a Markov chain of continuous time parameters. The results show the validity of this algorithm.
On cover times of Markov chains

Bruno Sericola

DOI: https://doi.org/10.1080/15326349.2024.2319201

2024-03-14

Stochastic Models

Abstract:We consider the cover time of a discrete-time homogenous Markov chain, which is the time needed by the Markov chain to visit all its states. We analyze both the distribution and the moments of the cover time, and we are interested in exact results instead of asymptotic values of the mean cover time, which are generally considered in the literature. We first obtain several general results on the hitting time and the cover time of a subset of the state space, both in terms of distribution and moments. These results are then applied to particular graphs, namely the generalized cycle graph, the complete graph, and the generalized path graph. They lead to recurrence or analytic relations for the distribution and the mean value of their cover times.

statistics & probability
The Computation of the Mean First Passage Times for Markov Chains

Jeffrey J Hunter

DOI: https://doi.org/10.1016/j.laa.2018.03.010

2018-03-16

Abstract:A survey of a variety of computational procedures for finding the mean first passage times in Markov chains is presented. The author recently developed a new accurate computational technique, an Extended GTH Procedure, Hunter (Special Matrices, 2016) similar to that developed by Kohlas (Zeit. fur Oper. Res., 1986). In addition, the author recently developed a variety of new perturbation techniques for finding key properties of Markov chains including finding the mean first passage times, Hunter (Linear Algebra and its Applications, 2016). These recently developed procedures are compared with other procedures including the standard matrix inversion technique using the fundamental matrix (Kemeny and Snell, 1960), some simple generalized matrix inverse techniques developed by Hunter (Asia Pacific J. Oper. Res., 2007), and some modifications to the FUND technique of Heyman (SIAM J Matrix Anal. and Appl., 1995). MATLAB is used to compute errors and estimate computation times when the techniques are used on some test problems that have been used in the literature together with some large sparse state-space cases. For accuracy a preference for the procedure of the author is exhibited for the test problems. However it appears that the procedure, as presented, requires longer computational times.

Numerical Analysis,Probability
The Derivation of Markov Chain Properties using Generalized Matrix Inverses

Jeffrey J. Hunter

DOI: https://doi.org/10.48550/arXiv.1112.3404

2011-12-15

Abstract:The analysis of many problems of interest associated with Markov chains, e.g. stationary distributions, moments of first passage time distributions and moments of occupation time random variables, involves the solution of a system of linear equations involving I - P, where P is the transition matrix of a finite, irreducible, discrete time Markov chain. Generalized inverses play an important role in the solution of such singular sets of equations. In this presentation we survey the application of generalized matrix inverses to the aforementioned problems.

Probability

Collective marks and first passage times

Higher Order Ergodic Markov Chains and First Passage Times

First Passage Times for Markov Renewal Processes and Applications

Waiting times and stopping probabilities for patterns in Markov chains

The First-Passage Times of Phase Semi-Markov Processes

First-passage times in renewal and nonrenewal systems

Exact analytic expressions for discrete first-passage time probability distributions in Markov networks

New First Passage Times and Their Distributions

A new method for first-passage time probability density function of structures

The First Passage Time Problem for Mixed-Exponential Jump Processes with Applications in Insurance and Finance

Higher-order statistics-based estimation of moments of first passage time for Markov chains in its reverse problem

Circulations of Markov-Chains with Continuous-Time and the Probability Interpretation of Some Determinants

First Passage Probabilities of One-Dimensional Diffusion Processes

On the Longest Run and the Waiting Time for the First Run in a Continuous Time Multi-State Markov Chain

The Distribution of Mixing Times in Markov Chains

Bounds on Fluctuations of First Passage Times for Counting Observables in Classical and Quantum Markov Processes

Semi-Markov Processes in Open Quantum Systems. III. Large Deviations of First Passage Time Statistics

Estimation Method and Its Properties of Moments of First Passage Time for Markov Chains in Reverse Problems

On cover times of Markov chains

The Computation of the Mean First Passage Times for Markov Chains

The Derivation of Markov Chain Properties using Generalized Matrix Inverses