Abstract:This paper introduces a novel algorithm to approximate the matrix with minimum nuclear norm among all matrices obeying a set of convex constraints. This problem may be understood as the convex relaxation of a rank minimization problem and arises in many important applications as in the task of recovering a large matrix from a small subset of its entries (the famous Netflix problem). Off-the-shelf algorithms such as interior point methods are not directly amenable to large problems of this kind with over a million unknown entries. This paper develops a simple first-order and easy-to-implement algorithm that is extremely efficient at addressing problems in which the optimal solution has low rank. The algorithm is iterative, produces a sequence of matrices $\{\boldsymbol{X}^k,\boldsymbol{Y}^k\}$, and at each step mainly performs a soft-thresholding operation on the singular values of the matrix $\boldsymbol{Y}^k$. There are two remarkable features making this attractive for low-rank matrix completion problems. The first is that the soft-thresholding operation is applied to a sparse matrix; the second is that the rank of the iterates $\{\boldsymbol{X}^k\}$ is empirically nondecreasing. Both these facts allow the algorithm to make use of very minimal storage space and keep the computational cost of each iteration low. On the theoretical side, we provide a convergence analysis showing that the sequence of iterates converges. On the practical side, we provide numerical examples in which $1,000\times1,000$ matrices are recovered in less than a minute on a modest desktop computer. We also demonstrate that our approach is amenable to very large scale problems by recovering matrices of rank about 10 with nearly a billion unknowns from just about 0.4% of their sampled entries. Our methods are connected with the recent literature on linearized Bregman iterations for $\ell_1$ minimization, and we develop a framework in which one can understand these algorithms in terms of well-known Lagrange multiplier algorithms.

Interlacing Polynomial Method for the Column Subset Selection Problem

Interlacing Polynomial Method for Matrix Approximation via Generalized Column and Row Selection

Fair Column Subset Selection

An Explicit Sampling Dependent Spectral Error Bound for Column Subset Selection

A sublinear-time randomized algorithm for column and row subset selection based on strong rank-revealing QR factorizations

An elementary approach to the problem of column selection in a rectangular matrix

Low Rank Approximation of Binary Matrices: Column Subset Selection and Generalizations

SPSD Matrix Approximation Vis Column Selection: Theories, Algorithms, and Extensions

Improved guarantees and a multiple-descent curve for Column Subset Selection and the Nyström method

Column and row subset selection using nuclear scores: algorithms and theory for Nyström approximation, CUR decomposition, and graph Laplacian reduction

New Subset Selection Algorithms for Low Rank Approximation: Offline and Online

On Socially Fair Low-Rank Approximation and Column Subset Selection

Adaptive Voronoi-based Column Selection Methods for Interpretable Dimensionality Reduction

Greedy Column Subset Selection: New Bounds and Distributed Algorithms

A preconditioner for solving linear programming problems with dense columns

On the Optimality of CVOD-based Column Selection

Low-Rank Matrix Iteration Using Polynomial-Filtered Subspace Extraction.

Projective convergence of columns for inhomogeneous products of matrices with nonnegative entries

A Singular Value Thresholding Algorithm for Matrix Completion

Sketched and Truncated Polynomial Krylov Subspace Methods: Matrix Sylvester Equations

Low Rank Matrix Completion via Robust Alternating Minimization in Nearly Linear Time