Exact analytic expressions for discrete first-passage time probability distributions in Markov networks

Jaroslav Albert

2024-03-21

Abstract:The first-passage time (FPT) is the time it takes a system variable to cross a given boundary for the first time. In the context of Markov networks, the FPT is the time a random walker takes to reach a particular node (target) by hopping from one node to another. If the walker pauses at each node for a period of time drawn from a continuous distribution, the FPT will be a continuous variable; if the pauses last exactly one unit of time, the FPT will be discrete and equal to the number of hops. We derive an exact analytical expression for the discrete first-passage time (DFPT) in Markov networks. Our approach is as follows: first, we divide each edge (connection between two nodes) of the network into $h$ unidirectional edges connecting a cascade of $h$ fictitious nodes and compute the continuous FPT (CFPT). Second, we set the transition rates along the edges to $h$, and show that as $h\to\infty$, the distribution of travel times between any two nodes of the original network approaches a delta function centered at 1, which is equivalent to pauses lasting 1 unit of time. Using this approach, we also compute the joint-probability distributions for the DFPT, the target node, and the node from which the target node was reached. A comparison with simulation confirms the validity of our approach.

Molecular Networks

What problem does this paper attempt to address?

This paper primarily addresses the problem of accurately calculating the discrete first-passage time (DFPT) distribution in a Markov network. In a Markov network, a random walker jumps from one node to another, and the time it takes to reach a specific target node for the first time is defined as the first-passage time. Typically, this time is difficult to analytically calculate, especially in complex networks.

The authors propose a novel approach by first decomposing each edge (the connection between nodes) into a series of unidirectional connected virtual nodes and calculating the continuous first-passage time (CFPT). Then, they set transition rates on these edges, such that as the transition rates tend to infinity, the travel time between nodes approximates a delta function centered around 1, which simulates staying at each node for one unit of time and yields the discrete first-passage time.

Through this approach, they not only obtain the exact analytical expression for DFPT but also compute the joint probability distribution of DFPT, the target node, and the starting node for reaching the target node. By comparing with simulation results, the effectiveness of this method is validated. The paper concludes by discussing the computational complexity of calculating the joint probability and suggesting that this method can be extended to other similar network systems, such as diffusing particles in a grid and Brownian motion.