Abstract:We study the matrix completion problem: an underlying matrix P is low rank, with incoherent singular vectors, and a random matrix A is equal to P on a (uniformly) random subset of entries of size dn . All other entries of A are equal to zero. The goal is to retrieve information on P from the observation of A . Let be the random matrix where each entry of A is multiplied by an independent -Bernoulli random variable with parameter 1/2. This paper is about when, how and why the non-Hermitian eigen-spectra of the matrices and captures more of the relevant information about the principal component structure of A than the eigen-spectra of and . We show that the eigenvalues of the asymmetric matrices and with modulus greater than a detection threshold are asymptotically equal to the eigenvalues of and and that the associated eigenvectors are aligned as well. The central surprise is that by intentionally inducing asymmetry and additional randomness via the matrix, we can extract more information than if we had worked with the singular value decomposition (SVD) of A . The associated detection threshold is asymptotically exact and is non-universal since it explicitly depends on the element-wise distribution of the underlying matrix P . We show that reliable, statistically optimal but not perfect matrix recovery, via a universal data-driven algorithm, is possible above this detection threshold using the information extracted from the asymmetric eigen-decompositions. Averaging the left and right eigenvectors provably improves estimation accuracy but not the detection threshold. Our results encompass the very sparse regime where d is of order 1 where matrix completion via the SVD of A fails or produces unreliable recovery. We define another variant of this asymmetric principal component analysis procedure that bypasses the randomization step and has a detection threshold that is smaller by a constant factor but with a computational cost that is larger by a polynomial factor of the number of observed entries. Both detection thresholds allow to go beyond the barrier due to the well-known information theoretical limit for exact matrix completion found in the literature.

Early stopping for statistical inverse problems via truncated SVD estimation

Early stopping for conjugate gradients in statistical inverse problems

Uncertainty quantification for iterative algorithms in linear models with application to early stopping

Perturbation Analysis of Randomized SVD and its Applications to High-dimensional Statistics

Stopping criterion for iterative regularization of large-scale ill-posed problems using the Picard parameter

Accelerated singular value thresholding for matrix completion.

Optimal Sparse Singular Value Decomposition for High-dimensional High-order Data

Modified Truncated Randomized Singular Value Decomposition (MTRSVD) Algorithms for Large Scale Discrete Ill-Posed Problems with General-Form Regularization

Early Stopping for Ensemble Kalman-Bucy Inversion

High Order Numerical Methods To Approximate The Singular Value Decomposition

Fast Updating Truncated SVD for Representation Learning with Sparse Matrices

Generalizing the SVD of a matrix under non-standard inner product and its applications to linear ill-posed problems

Incremental algorithms for truncated higher-order singular value decompositions

Analysis of the rSVDdpd Algorithm: A Robust Singular Value Decomposition Method using Density Power Divergence

A Sparse SVD Method for High-dimensional Data

Regularization Properties of Krylov Iterative Solvers CGME and LSMR for Linear Discrete Ill-Posed Problems with an Application to Truncated Randomized SVDs

Row-aware Randomized SVD with applications

Detection Thresholds in Very Sparse Matrix Completion

Stopping criteria for nonlinear variational problems: iterative approach

SMSSVD - SubMatrix Selection Singular Value Decomposition

Efficient Statistics With Unknown Truncation, Polynomial Time Algorithms, Beyond Gaussians