Abstract:Randomized singular value decomposition (RSVD) is a class of computationally efficient algorithms for computing the truncated SVD of large data matrices. Given a $n \times n$ symmetric matrix $\mathbf{M}$, the prototypical RSVD algorithm outputs an approximation of the $k$ leading singular vectors of $\mathbf{M}$ by computing the SVD of $\mathbf{M}^{g} \mathbf{G}$; here $g \geq 1$ is an integer and $\mathbf{G} \in \mathbb{R}^{n \times k}$ is a random Gaussian sketching matrix. In this paper we study the statistical properties of RSVD under a general "signal-plus-noise" framework, i.e., the observed matrix $\hat{\mathbf{M}}$ is assumed to be an additive perturbation of some true but unknown signal matrix $\mathbf{M}$. We first derive upper bounds for the $\ell_2$ (spectral norm) and $\ell_{2\to\infty}$ (maximum row-wise $\ell_2$ norm) distances between the approximate singular vectors of $\hat{\mathbf{M}}$ and the true singular vectors of the signal matrix $\mathbf{M}$. These upper bounds depend on the signal-to-noise ratio (SNR) and the number of power iterations $g$. A phase transition phenomenon is observed in which a smaller SNR requires larger values of $g$ to guarantee convergence of the $\ell_2$ and $\ell_{2\to\infty}$ distances. We also show that the thresholds for $g$ where these phase transitions occur are sharp whenever the noise matrices satisfy a certain trace growth condition. Finally, we derive normal approximations for the row-wise fluctuations of the approximate singular vectors and the entrywise fluctuations of the approximate matrix. We illustrate our theoretical results by deriving nearly-optimal performance guarantees for RSVD when applied to three statistical inference problems, namely, community detection, matrix completion, and principal component analysis with missing data.

Application of the projective truncation and randomized singular value decomposition to a higher dimension

Randomized higher-order tensor renormalization group

Hot-SVD: Higher-Order t-Singular Value Decomposition for Tensors based on Tensor-Tensor Product

Initial tensor construction and dependence of the tensor renormalization group on initial tensors

All-mode Renormalization for Tensor Network with Stochastic Noise

Randomized Algorithms for Computation of Tucker Decomposition and Higher Order SVD (HOSVD)

Tensor Random Projection for Low Memory Dimension Reduction

Perturbation Analysis of Randomized SVD and its Applications to High-dimensional Statistics

Practical sketching‐based randomized tensor ring decomposition

Mixed-Precision Random Projection for RandNLA on Tensor Cores

Nonconvex Robust High-Order Tensor Completion Using Randomized Low-Rank Approximation

Revisiting High-Order Tensor Singular Value Decomposition From Basic Element Perspective

A randomized singular value decomposition for third-order oriented tensors

Incremental algorithms for truncated higher-order singular value decompositions

Renormalization group on a triad network

Efficient Alternating Least Squares Algorithms for Truncated HOSVD of Higher-Order Tensors.

An Efficient Randomized Fixed-Precision Algorithm for Tensor Singular Value Decomposition

Parallel Randomized Tucker Decomposition Algorithms

Randomized algorithms for computing the tensor train approximation and their applications

Generalized Higher-Order Tensor Decomposition via Parallel ADMM