Abstract:We study the problem of learning a partially observed matrix under the low rank assumption in the presence of fully observed side information that depends linearly on the true underlying matrix. This problem consists of an important generalization of the Matrix Completion problem, a central problem in Statistics, Operations Research and Machine Learning, that arises in applications such as recommendation systems, signal processing, system identification and image denoising. We formalize this problem as an optimization problem with an objective that balances the strength of the fit of the reconstruction to the observed entries with the ability of the reconstruction to be predictive of the side information. We derive a mixed-projection reformulation of the resulting optimization problem and present a strong semidefinite cone relaxation. We design an efficient, scalable alternating direction method of multipliers algorithm that produces high quality feasible solutions to the problem of interest. Our numerical results demonstrate that in the small rank regime ($k \leq 15$), our algorithm outputs solutions that achieve on average $79\%$ lower objective value and $90.1\%$ lower $\ell_2$ reconstruction error than the solutions returned by the best performing benchmark method on synthetic data. The runtime of our algorithm is competitive with and often superior to that of the benchmark methods. Our algorithm is able to solve problems with $n = 10000$ rows and $m = 10000$ columns in less than a minute. On large scale real world data, our algorithm produces solutions that achieve $67\%$ lower out of sample error than benchmark methods in $97\%$ less execution time.

Low-Rank Matrix Estimation from Rank-One Projections by Unlifted Convex Optimization

Improved Algorithms for Matrix Recovery from Rank-One Projections

Convergence of projected Landweber iteration for matrix rank minimization

Guarantees of Riemannian Optimization for Low Rank Matrix Recovery

Low-rank matrix recovery with non-quadratic loss: projected gradient method and regularity projection oracle

Beyond Procrustes: Balancing-Free Gradient Descent for Asymmetric Low-Rank Matrix Sensing

Projected Gradient Descent Algorithm for Low-Rank Matrix Estimation

Fast low-rank estimation by projected gradient descent: General statistical and algorithmic guarantees

Low-rank matrix estimation via nonconvex spectral regularized methods in errors-in-variables matrix regression

Can Learning Be Explained By Local Optimality In Low-rank Matrix Recovery?

Exploiting the Structure Effectively and Efficiently in Low Rank Matrix Recovery.

Fast Low-Rank Matrix Learning with Nonconvex Regularization

Near-optimal estimation of simultaneously sparse and low-rank matrices from nested linear measurements

Nonconvex Low-Rank Matrix Recovery with Arbitrary Outliers via Median-Truncated Gradient Descent

A Unified Computational and Statistical Framework for Nonconvex Low-rank Matrix Estimation

Predictive Low Rank Matrix Learning under Partial Observations: Mixed-Projection ADMM

Adaptive Rank-One Matrix Completion using Sum of Outer Products

Sketching low-rank matrices with a shared column space by convex programming

Robust Recovery of Low-Rank Matrices Via Non-Convex Optimization

Estimation of high-dimensional low-rank matrices

A NONCONVEX ALGORITHM FOR SIMULTANEOUSLY SPARSE AND LOW-RANK MATRIX RECONSTRUCTION