Abstract:Randomized singular value decomposition (RSVD) is a class of computationally efficient algorithms for computing the truncated SVD of large data matrices. Given a $n \times n$ symmetric matrix $\mathbf{M}$, the prototypical RSVD algorithm outputs an approximation of the $k$ leading singular vectors of $\mathbf{M}$ by computing the SVD of $\mathbf{M}^{g} \mathbf{G}$; here $g \geq 1$ is an integer and $\mathbf{G} \in \mathbb{R}^{n \times k}$ is a random Gaussian sketching matrix. In this paper we study the statistical properties of RSVD under a general "signal-plus-noise" framework, i.e., the observed matrix $\hat{\mathbf{M}}$ is assumed to be an additive perturbation of some true but unknown signal matrix $\mathbf{M}$. We first derive upper bounds for the $\ell_2$ (spectral norm) and $\ell_{2\to\infty}$ (maximum row-wise $\ell_2$ norm) distances between the approximate singular vectors of $\hat{\mathbf{M}}$ and the true singular vectors of the signal matrix $\mathbf{M}$. These upper bounds depend on the signal-to-noise ratio (SNR) and the number of power iterations $g$. A phase transition phenomenon is observed in which a smaller SNR requires larger values of $g$ to guarantee convergence of the $\ell_2$ and $\ell_{2\to\infty}$ distances. We also show that the thresholds for $g$ where these phase transitions occur are sharp whenever the noise matrices satisfy a certain trace growth condition. Finally, we derive normal approximations for the row-wise fluctuations of the approximate singular vectors and the entrywise fluctuations of the approximate matrix. We illustrate our theoretical results by deriving nearly-optimal performance guarantees for RSVD when applied to three statistical inference problems, namely, community detection, matrix completion, and principal component analysis with missing data.

Modifying the Generalized Singular Value Decomposition with Application with Application in Direction-of-arrival Finding

A Modified Gram-Schmidt-Based Downdating Technique for Ulv Decompositions with Applications to Recursive Tls Problems

On Choices of Formulations of Computing the Generalized Singular Value Decomposition of a Large Matrix Pair.

Incremental algorithms for truncated higher-order singular value decompositions

Two-sided uniformly randomized GSVD for large-scale discrete ill-posed problems with Tikhonov regularizations

Perturbation Analysis of Randomized SVD and its Applications to High-dimensional Statistics

Vandermonde Decomposition of Coprime Coarray Covariance Matrix for DOA Estimation

Fast Updating Truncated SVD for Representation Learning with Sparse Matrices

Adaptive Algorithms for Generalized Eigenvalue Decomposition with A Nonquadratic Criterion

The Singular Value Decomposition, Applications and Beyond.

On Modifying the Variational Quantum Singular Value Decomposition Algorithm

Perturbation Analysis for Subspace Decomposition with Applications in Subspace-Based Algorithms

The Joint Bidiagonalization Method for Large GSVD Computations in Finite Precision.

Characterizing GSVD by singular value expansion of linear operators and its computation

Generalizing the SVD of a matrix under non-standard inner product and its applications to linear ill-posed problems

Row-aware Randomized SVD with applications

A Higher-Order Generalized Singular Value Decomposition for Rank Deficient Matrices

Tikhonov Regularization and Randomized GSVD.

Towards a more robust algorithm for computing the restricted singular value decomposition

Low Complexity Adaptive Algorithm for Generalized Eigenvalue Decomposition

Efficient Bounds and Estimates for Canonical Angles in Randomized Subspace Approximations