Abstract:This work is concerned with computing low-rank approximations of a matrix function $f(A)$ for a large symmetric positive semi-definite matrix $A$, a task that arises in, e.g., statistical learning and inverse problems. The application of popular randomized methods, such as the randomized singular value decomposition or the Nyström approximation, to $f(A)$ requires multiplying $f(A)$ with a few random vectors. A significant disadvantage of such an approach, matrix-vector products with $f(A)$ are considerably more expensive than matrix-vector products with $A$, even when carried out only approximately via, e.g., the Lanczos method. In this work, we present and analyze funNyström, a simple and inexpensive method that constructs a low-rank approximation of $f(A)$ directly from a Nyström approximation of $A$, completely bypassing the need for matrix-vector products with $f(A)$. It is sensible to use funNyström whenever $f$ is monotone and satisfies $f(0) = 0$. Under the stronger assumption that $f$ is operator monotone, which includes the matrix square root $A^{1/2}$ and the matrix logarithm $\log(I+A)$, we derive probabilistic bounds for the error in the Frobenius, nuclear, and operator norms. These bounds confirm the numerical observation that funNyström tends to return an approximation that compares well with the best low-rank approximation of $f(A)$. Furthermore, compared to existing methods, funNyström requires significantly fewer matrix-vector products with $A$ to obtain a low-rank approximation of $f(A)$, without sacrificing accuracy or reliability. Our method is also of interest when estimating quantities associated with $f(A)$, such as the trace or the diagonal entries of $f(A)$. In particular, we propose and analyze funNyström++, a combination of funNyström with the recently developed Hutch++ method for trace estimation.

Making Large-Scale Nyström Approximation Possible

Making the Nyström method highly accurate for low-rank approximations

Large-scale Eigenvector Approximation Via Hilbert Space Embedding Nyström

Improving the Modified Nyström Method Using Spectral Shifting

A Multilinear Nyström Algorithm for Low-Rank Approximation of Tensors in Tucker Format

Randomized Nyström approximation of non-negative self-adjoint operators

Robust SVD Made Easy: A fast and reliable algorithm for large-scale data analysis

Randomized low-rank approximation for symmetric indefinite matrices

Reduced Heteroscedasticity Linear Regression for Nyström Approximation

Non-uniform Nyström approximation for sparse kernel regression: Theoretical analysis and experimental evaluation

A generalized Nystrom method with column sketching for low-rank approximation of nonsymmetric matrices

Towards More Efficient SPSD Matrix Approximation and CUR Matrix Decomposition

Randomized low-rank approximation of monotone matrix functions

Accelerated singular value thresholding for matrix completion.

Nyström Approximation with Nonnegative Matrix Factorization

Single-pass Nyström approximation in mixed precision

Approximating Element-Wise Functions of Matrix with Improved Streaming Randomized SVD

Column and row subset selection using nuclear scores: algorithms and theory for Nyström approximation, CUR decomposition, and graph Laplacian reduction

Improving CUR Matrix Decomposition and the Nyström Approximation Via Adaptive Sampling

On the Unreasonable Effectiveness of Single Vector Krylov Methods for Low-Rank Approximation