Abstract:Markov chains that describe interacting subsystems suffer, on the one hand, from state space explosion but lead, on the other hand, to highly structured matrices. In this work, we propose a novel tensor-based algorithm to address such tensor structured Markov chains. Our algorithm combines a tensorized multigrid method with AMEn, an optimization-based low-rank tensor solver, for addressing coarse grid problems. Numerical experiments demonstrate that this combination overcomes the limitations incurred when using each of the two methods individually. As a consequence, Markov chain models of unprecedented size from a variety of applications can be addressed.
What problem does this paper attempt to address?
The main problem that this paper attempts to solve is the numerical calculation of the steady - state distribution of large - scale continuous - time Markov chains. Specifically, the author focuses on Markov chains with a tensor structure. This type of model describes multiple interacting subsystems, which leads to the problem of state - space explosion, but also generates highly - structured matrices at the same time.
### Problem Background
Mathematically, the task of calculating the steady - state distribution can be reduced to solving a system of linear equations:
\[ A\mathbf{x} = 0 \quad \text{with} \quad \mathbf{1}^T \mathbf{x} = 1, \]
where \( A \) is the generator matrix of the Markov chain, and \( \mathbf{1} \) is a vector of all ones. The matrix \( A \) is asymmetric and satisfies \( \mathbf{1}^T A = 0 \). For an irreducible \( A \), this equation has a unique solution.
### Challenges
1. **State - Space Explosion**: When the number \( d \) of subsystems increases, the size \( n = n_1 n_2 \cdots n_d \) of the state - space will grow exponentially, which makes storage and computation very difficult.
2. **Limitations of Existing Methods**:
- **Standard Iterative Solvers**: Unable to handle larger \( d \), because it is necessary to store vectors of length \( n \).
- **Tensorized Multigrid Methods**: Although it can reduce the pattern size \( n_k \) and the condition number, the sub - problems on the coarse grid are still difficult to solve.
- **Low - Rank Tensor Methods**: Such as the AMEn method, can handle larger \( d \) to a certain extent, but for sub - problems on the coarse grid, its convergence is poor.
### Solutions
To solve the above problems, the author proposes a new algorithm that combines the tensorized multigrid method and the low - rank tensor method. Specifically:
- The **tensorized multigrid method** is used to reduce the pattern size \( n_k \) and the condition number, which is beneficial for the application of the low - rank tensor method.
- The **AMEn method** is used to solve the sub - problems on the coarse grid and find approximate solutions in the low - rank tensor format through the alternating optimization technique.
### Main Contributions
By combining the advantages of these two methods, the new algorithm can overcome the limitations when they are used separately, and thus can solve hitherto unprecedented large - scale Markov chain models. Numerical experiments show that this combined method performs well in multiple application scenarios and significantly improves computational efficiency and accuracy.
In conclusion, this paper aims to solve the state - space explosion problem in the calculation of the steady - state distribution of large - scale tensor - structured Markov chains through innovative algorithm design, providing an effective tool for research in related fields.