Abstract:Various iterative eigenvalue solvers have been developed to compute parts of the spectrum for a large sparse matrix, including the power method, Krylov subspace methods, contour integral methods, and preconditioned solvers such as the so called LOBPCG method. All of these solvers rely on random matrices to determine, e.g., starting vectors that have, with high probability, a non-negligible overlap with the eigenvectors of interest. For this purpose, a safe and common choice are unstructured Gaussian random matrices. In this work, we investigate the use of random Khatri-Rao products in eigenvalue solvers. On the one hand, we establish a novel subspace embedding property that provides theoretical justification for the use of such structured random matrices. On the other hand, we highlight the potential algorithmic benefits when solving eigenvalue problems with Kronecker product structure, as they arise frequently from the discretization of eigenvalue problems for differential operators on tensor product domains. In particular, we consider the use of random Khatri-Rao products within a contour integral method and LOBPCG. Numerical experiments indicate that the gains for the contour integral method strongly depend on the ability to efficiently and accurately solve (shifted) matrix equations with low-rank right-hand side. The flexibility of LOBPCG to directly employ preconditioners makes it easier to benefit from Khatri-Rao product structure, at the expense of having less theoretical justification.
What problem does this paper attempt to address?
### What problem does this paper attempt to solve?
This paper aims to explore the application of the randomized Khatri–Rao product in eigenvalue solvers, especially for the eigenvalue computation problems of large - scale sparse matrices. Specifically, the authors focus on the following aspects:
1. **Theoretical verification**:
- A new subspace embedding (OSE) property of the randomized Khatri–Rao product is established, providing a theoretical basis for using this structured random matrix. Equation (5) shows the required number of samples \( \ell \), to ensure that the OSE property is satisfied:
\[
\ell \geq C \left( k^{3/2} \varepsilon^{-2} + k \log(1/\delta) \varepsilon^{-2} + k^{1/2} \log^2(1/\delta) \varepsilon^{-1} \right)
\]
where \( C=(2000 e^4)^2 \).
2. **Algorithm efficiency**:
- The potential algorithmic advantages of using the randomized Khatri–Rao product in solving eigenvalue problems are studied, especially when dealing with matrices with Kronecker product structures. Such structures are common in problems after the discretization of differential operators on tensor - product domains.
- In particular, in the contour integral method and LOBPCG (locally optimal block preconditioned conjugate gradient method), it is shown how to use the randomized Khatri–Rao product to improve computational efficiency. For example, in the contour integral method, the calculation of equation (2) can be transformed into solving a series of Sylvester equations with low - rank right - hand sides, thus solving the problem more efficiently.
3. **Numerical experiments**:
- Through numerical experiments, it is verified that the accuracy of computing invariant subspaces using the randomized Khatri–Rao product is comparable to that using unstructured Gaussian random matrices. Experiments show that although for some specific cases (such as dealing with rank - one vectors), the randomized Khatri–Rao product may require more samples, the overall performance is still good.
- For the LOBPCG method, experiments show that efficient computation can be achieved by limiting the rank of iterations while maintaining convergence.
### Main contributions
- The OSE property of the randomized Khatri–Rao product is proposed, and a clear lower bound on the number of samples is given.
- The applications of the randomized Khatri–Rao product in the contour integral method and LOBPCG are explored, showing its superiority in dealing with matrices with Kronecker product structures.
- The theoretical results are verified through numerical experiments, and the effectiveness of these methods in practical applications is demonstrated.
In summary, this paper not only theoretically proves the effectiveness of the randomized Khatri–Rao product, but also shows its practical application value in eigenvalue solving through specific algorithm improvements and experimental verification.