Abstract:Randomized singular value decomposition (RSVD) is a class of computationally efficient algorithms for computing the truncated SVD of large data matrices. Given a $n \times n$ symmetric matrix $\mathbf{M}$, the prototypical RSVD algorithm outputs an approximation of the $k$ leading singular vectors of $\mathbf{M}$ by computing the SVD of $\mathbf{M}^{g} \mathbf{G}$; here $g \geq 1$ is an integer and $\mathbf{G} \in \mathbb{R}^{n \times k}$ is a random Gaussian sketching matrix. In this paper we study the statistical properties of RSVD under a general "signal-plus-noise" framework, i.e., the observed matrix $\hat{\mathbf{M}}$ is assumed to be an additive perturbation of some true but unknown signal matrix $\mathbf{M}$. We first derive upper bounds for the $\ell_2$ (spectral norm) and $\ell_{2\to\infty}$ (maximum row-wise $\ell_2$ norm) distances between the approximate singular vectors of $\hat{\mathbf{M}}$ and the true singular vectors of the signal matrix $\mathbf{M}$. These upper bounds depend on the signal-to-noise ratio (SNR) and the number of power iterations $g$. A phase transition phenomenon is observed in which a smaller SNR requires larger values of $g$ to guarantee convergence of the $\ell_2$ and $\ell_{2\to\infty}$ distances. We also show that the thresholds for $g$ where these phase transitions occur are sharp whenever the noise matrices satisfy a certain trace growth condition. Finally, we derive normal approximations for the row-wise fluctuations of the approximate singular vectors and the entrywise fluctuations of the approximate matrix. We illustrate our theoretical results by deriving nearly-optimal performance guarantees for RSVD when applied to three statistical inference problems, namely, community detection, matrix completion, and principal component analysis with missing data.

An exact $\sinΘ$ formula for matrix perturbation analysis and its applications

Matrix perturbation analysis of methods for extracting singular values from approximate singular subspaces

Matrices with Gaussian Noise: Optimal Estimates for Singular Subspace Perturbation

Perturbation expansions and error bounds for the truncated singular value decomposition

Perturbation Analysis for Subspace Decomposition with Applications in Subspace-Based Algorithms

Matrix Perturbation: Davis-Kahan in the Infinity Norm

A Riemann--Hilbert approach to the perturbation theory for orthogonal polynomials: Applications to numerical linear algebra and random matrix theory

New matrix perturbation bounds via combinatorial expansion I: Perturbation of eigenspaces

Analysis of singular subspaces under random perturbations

A Schatten-$q$ Low-rank Matrix Perturbation Analysis via Perturbation Projection Error Bound

Relative perturbation bounds for the eigenvalues of diagonalizable and singular matrices – Application of perturbation theory for simple invariant subspaces

Singular Subspace Perturbation Bounds via Rectangular Random Matrix Diffusions

A note on perturbation analysis for T-product based tensor singular values

A universal matrix perturbation technique for structural dynamic modification using singular value decomposition

On Stewart's Perturbation Theorem for SVD

Perturbation Analysis of Randomized SVD and its Applications to High-dimensional Statistics

On Eigenvector Approximation of Diagonalizable Random Matrices with Random Perturbations: Properties and Applications

Perturbation and Inverse Problems of Stochastic Matrices

Probabilistic perturbation bounds of matrix decompositions

Perturbation Analysis of A_T,S^(2) on Banach Spaces

Perturbation of the Drazin Inverse for Matrices with Equal Eigenprojections at Zero