Abstract:This work is concerned with computing low-rank approximations of a matrix function $f(A)$ for a large symmetric positive semi-definite matrix $A$, a task that arises in, e.g., statistical learning and inverse problems. The application of popular randomized methods, such as the randomized singular value decomposition or the Nyström approximation, to $f(A)$ requires multiplying $f(A)$ with a few random vectors. A significant disadvantage of such an approach, matrix-vector products with $f(A)$ are considerably more expensive than matrix-vector products with $A$, even when carried out only approximately via, e.g., the Lanczos method. In this work, we present and analyze funNyström, a simple and inexpensive method that constructs a low-rank approximation of $f(A)$ directly from a Nyström approximation of $A$, completely bypassing the need for matrix-vector products with $f(A)$. It is sensible to use funNyström whenever $f$ is monotone and satisfies $f(0) = 0$. Under the stronger assumption that $f$ is operator monotone, which includes the matrix square root $A^{1/2}$ and the matrix logarithm $\log(I+A)$, we derive probabilistic bounds for the error in the Frobenius, nuclear, and operator norms. These bounds confirm the numerical observation that funNyström tends to return an approximation that compares well with the best low-rank approximation of $f(A)$. Furthermore, compared to existing methods, funNyström requires significantly fewer matrix-vector products with $A$ to obtain a low-rank approximation of $f(A)$, without sacrificing accuracy or reliability. Our method is also of interest when estimating quantities associated with $f(A)$, such as the trace or the diagonal entries of $f(A)$. In particular, we propose and analyze funNyström++, a combination of funNyström with the recently developed Hutch++ method for trace estimation.

On the sum-of-products to product-of-sums transformation between analytical low-rank approximations in finite basis representation

A multilinear Nyström algorithm for low-rank approximation of tensors in Tucker format

Algorithm-agnostic low-rank approximation of operator monotone matrix functions

Higher-order principal component analysis for the approximation of tensors in tree-based low-rank formats

Low Rank Symmetric Tensor Approximations

Preconditioned low-rank Riemannian optimization for linear systems with tensor product structure

Efficient Alternating Least Squares Algorithms for Low Multilinear Rank Approximation of Tensors

A low-rank isogeometric solver based on Tucker tensors

Calculating vibrational spectra with sum of product basis functions without storing full-dimensional vectors or matrices

An Iterative Reweighted Method for Tucker Decomposition of Incomplete Multiway Tensors

On the Global Convergence of the Alternating Least Squares Method for Rank-One Approximation to Generic Tensors

Low-rank approximate inverse for preconditioning tensor-structured linear systems

Some Tucker-like approximations based on the modal semi-tensor product

CP decomposition and low-rank approximation of antisymmetric tensors

Randomized low-rank approximation of monotone matrix functions

On low-rank approximability of solutions to high-dimensional operator equations and eigenvalue problems

Fast and Accurate Randomized Algorithms for Low-rank Tensor Decompositions

Efficient randomized tensor-based algorithms for function approximation and low-rank kernel interactions

A sequential multilinear Nyström algorithm for streaming low-rank approximation of tensors in Tucker format

Efficient Tensor-Product Spectral-Element Operators with the Summation-by-Parts Property on Curved Triangles and Tetrahedra