Abstract:This paper introduces a novel algorithm to approximate the matrix with minimum nuclear norm among all matrices obeying a set of convex constraints. This problem may be understood as the convex relaxation of a rank minimization problem and arises in many important applications as in the task of recovering a large matrix from a small subset of its entries (the famous Netflix problem). Off-the-shelf algorithms such as interior point methods are not directly amenable to large problems of this kind with over a million unknown entries. This paper develops a simple first-order and easy-to-implement algorithm that is extremely efficient at addressing problems in which the optimal solution has low rank. The algorithm is iterative, produces a sequence of matrices $\{\boldsymbol{X}^k,\boldsymbol{Y}^k\}$, and at each step mainly performs a soft-thresholding operation on the singular values of the matrix $\boldsymbol{Y}^k$. There are two remarkable features making this attractive for low-rank matrix completion problems. The first is that the soft-thresholding operation is applied to a sparse matrix; the second is that the rank of the iterates $\{\boldsymbol{X}^k\}$ is empirically nondecreasing. Both these facts allow the algorithm to make use of very minimal storage space and keep the computational cost of each iteration low. On the theoretical side, we provide a convergence analysis showing that the sequence of iterates converges. On the practical side, we provide numerical examples in which $1,000\times1,000$ matrices are recovered in less than a minute on a modest desktop computer. We also demonstrate that our approach is amenable to very large scale problems by recovering matrices of rank about 10 with nearly a billion unknowns from just about 0.4% of their sampled entries. Our methods are connected with the recent literature on linearized Bregman iterations for $\ell_1$ minimization, and we develop a framework in which one can understand these algorithms in terms of well-known Lagrange multiplier algorithms.

A Designed Thresholding Operator for Low-Rank Matrix Completion

Adaptive iterative singular value thresholding algorithm to low-rank matrix recovery

Efficient Completion for Corrupted Low-Rank Images Via Alternating Direction Method

Normalized Iterative Hard Thresholding for Matrix Completion

Non-Local Image Inpainting Using Low-Rank Matrix Completion: Non-Local Image Inpainting Using Low-Rank Matrix Completion

Convergence of projected Landweber iteration for matrix rank minimization

On the Schatten P-Quasi-norm Minimization for Low-Rank Matrix Recovery

CONVERGENCE AND STABILITY OF ITERATIVELY REWEIGHTED LEAST SQUARES FOR LOW-RANK MATRIXRECOVERY

A Singular Value Thresholding Algorithm for Matrix Completion

Accelerated singular value thresholding for matrix completion.

Projected Gradient Descent Algorithm for Low-Rank Matrix Estimation

Minimum $n$-Rank Approximation via Iterative Hard Thresholding

Accelerating Ill-Conditioned Low-Rank Matrix Estimation via Scaled Gradient Descent

Guarantees of Riemannian Optimization for Low Rank Matrix Recovery

Iterative Methods Based on Soft Thresholding of Hierarchical Tensors

Adaptive Rank-One Matrix Completion using Sum of Outer Products

Iterative thresholding algorithm based on non-convex method for modified lp-norm regularization minimization

Low rank matrix completion by alternating steepest descent methods

Iterative Singular Tube Hard Thresholding Algorithms for Tensor Recovery

A Nearly Unbiased Matrix Completion Approach.

Low Rank Vectorized Hankel Lift for Matrix Recovery via Fast Iterative Hard Thresholding