A note on discrete-time markov chains

XU YIMING
2020-01-01
Abstract:P (Xt = it|Xk = ik, 0 ≤ k ≤ t− 1) = P (Xt = it|Xt−1 = it−1) = p it−1it . If p it−1it is independent of t, we say that the Markov chain is time-homogeneous. In this case we use the transition matrix P ∈ [0, 1]|S|×|S| to store pij . All the Markov chains in the following discussion are assumed to be time-homogeneous. A Markov chain is uniquely determined by its initial distribution and transition matrix, based on which one can define a few fundamental concepts. States i, j ∈ S are said to be in the same communicating class if from either state there exists a path to the other. It is easy to check that the communicating class gives a relation. Most properties of the Markov chain to be discussed later are communicating class properties. For i, j ∈ S and m ∈ N, define the m-th return time to i started at j as
What problem does this paper attempt to address?