Abstract:Randomized singular value decomposition (RSVD) is a class of computationally efficient algorithms for computing the truncated SVD of large data matrices. Given a $n \times n$ symmetric matrix $\mathbf{M}$, the prototypical RSVD algorithm outputs an approximation of the $k$ leading singular vectors of $\mathbf{M}$ by computing the SVD of $\mathbf{M}^{g} \mathbf{G}$; here $g \geq 1$ is an integer and $\mathbf{G} \in \mathbb{R}^{n \times k}$ is a random Gaussian sketching matrix. In this paper we study the statistical properties of RSVD under a general "signal-plus-noise" framework, i.e., the observed matrix $\hat{\mathbf{M}}$ is assumed to be an additive perturbation of some true but unknown signal matrix $\mathbf{M}$. We first derive upper bounds for the $\ell_2$ (spectral norm) and $\ell_{2\to\infty}$ (maximum row-wise $\ell_2$ norm) distances between the approximate singular vectors of $\hat{\mathbf{M}}$ and the true singular vectors of the signal matrix $\mathbf{M}$. These upper bounds depend on the signal-to-noise ratio (SNR) and the number of power iterations $g$. A phase transition phenomenon is observed in which a smaller SNR requires larger values of $g$ to guarantee convergence of the $\ell_2$ and $\ell_{2\to\infty}$ distances. We also show that the thresholds for $g$ where these phase transitions occur are sharp whenever the noise matrices satisfy a certain trace growth condition. Finally, we derive normal approximations for the row-wise fluctuations of the approximate singular vectors and the entrywise fluctuations of the approximate matrix. We illustrate our theoretical results by deriving nearly-optimal performance guarantees for RSVD when applied to three statistical inference problems, namely, community detection, matrix completion, and principal component analysis with missing data.

Singular-value statistics of directed random graphs

Singular Value Approximation and Sparsifying Random Walks on Directed Graphs

Singular-Value Statistics of Non-Hermitian Random Matrices and Open Quantum Systems

A Krylov space approach to Singular Value Decomposition in non-Hermitian systems

Perturbation Analysis of Randomized SVD and its Applications to High-dimensional Statistics

Non-asymptotic theory of random matrices: extreme singular values

Distributions of singular values for some random matrices

Singular value distribution of dense random matrices with block Markovian dependence

Universality of the least singular value and singular vector delocalisation for Lévy non-symmetric random matrices

Singularity degree of structured random matrices

Large deviations for the largest singular value of sparse non-Hermitian matrices

A Complex Network Analysis on The Eigenvalue Spectra of Random Spin Systems

Critical statistics for non-Hermitian matrices

Usefulness of signed eigenvalue/vector distributions of random tensors

Critical Statistics in Quantum Chaos and Calogero-Sutherland Model at Finite Temperature

Unfolding a composed ensemble of energy spectra using singular value decomposition

Universality for random matrices with an edge spectrum singularity

The least singular value of a random symmetric matrix

Parametric Spectral Statistics in Unitary Random Matrix Ensembles: From Distribution Functions to Intra-Level Correlations