Abstract:Randomized singular value decomposition (RSVD) is a class of computationally efficient algorithms for computing the truncated SVD of large data matrices. Given a $n \times n$ symmetric matrix $\mathbf{M}$, the prototypical RSVD algorithm outputs an approximation of the $k$ leading singular vectors of $\mathbf{M}$ by computing the SVD of $\mathbf{M}^{g} \mathbf{G}$; here $g \geq 1$ is an integer and $\mathbf{G} \in \mathbb{R}^{n \times k}$ is a random Gaussian sketching matrix. In this paper we study the statistical properties of RSVD under a general "signal-plus-noise" framework, i.e., the observed matrix $\hat{\mathbf{M}}$ is assumed to be an additive perturbation of some true but unknown signal matrix $\mathbf{M}$. We first derive upper bounds for the $\ell_2$ (spectral norm) and $\ell_{2\to\infty}$ (maximum row-wise $\ell_2$ norm) distances between the approximate singular vectors of $\hat{\mathbf{M}}$ and the true singular vectors of the signal matrix $\mathbf{M}$. These upper bounds depend on the signal-to-noise ratio (SNR) and the number of power iterations $g$. A phase transition phenomenon is observed in which a smaller SNR requires larger values of $g$ to guarantee convergence of the $\ell_2$ and $\ell_{2\to\infty}$ distances. We also show that the thresholds for $g$ where these phase transitions occur are sharp whenever the noise matrices satisfy a certain trace growth condition. Finally, we derive normal approximations for the row-wise fluctuations of the approximate singular vectors and the entrywise fluctuations of the approximate matrix. We illustrate our theoretical results by deriving nearly-optimal performance guarantees for RSVD when applied to three statistical inference problems, namely, community detection, matrix completion, and principal component analysis with missing data.

A stochastic perturbation analysis of the QR decomposition and its applications

New rigorous perturbation bounds for the LU and QR factorizations

Perturbation analysis for the hyperbolic QR factorization

Perturbation analysis for the symplectic QR factorization

Probabilistic perturbation bounds of matrix decompositions

Improved Rigorous Perturbation Bounds for the LU and QR Factorizations

Two-sided Generalized Hyperbolic QR Factorization and Its Perturbation Analysis

PERTURBATION THEORY FOR THE LU AND QR FACTORIZATIONS

Perturbation Analysis for Qr Factorization of Unitary-Symmetric Matrix

Perturbation Analysis for Subspace Decomposition with Applications in Subspace-Based Algorithms

A Schatten-$q$ Low-rank Matrix Perturbation Analysis via Perturbation Projection Error Bound

Perturbation expansions and error bounds for the truncated singular value decomposition

Perturbation Analysis of Randomized SVD and its Applications to High-dimensional Statistics

Tensor LU and QR Decompositions and Their Randomized Algorithms

Perturbation and Inverse Problems of Stochastic Matrices

Perturbation Analysis and Randomized Algorithms for Large-Scale Total Least Squares Problems

On Mixed and Componentwise Condition Numbers for Hyperbolic Qr Factorization

Quantitative Analysis for Perturbed Abstract Inequality Systems in Banach Spaces

A QLP Decomposition via Randomization.

Probabilistic Analysis of Least Squares, Orthogonal Projection, and QR Factorization Algorithms Subject to Gaussian Noise

Perturbation Analysis in Verification of Discrete-Time Markov Chains.