Abstract:Suppose that we observe entries or, more generally, linear combinations of entries of an unknown m×T-matrix A corrupted by noise. We are particularly interested in the high-dimensional setting where the number mT of unknown entries can be much larger than the sample size N. Motivated by several applications, we consider estimation of matrix A under the assumption that it has small rank. This can be viewed as dimension reduction or sparsity assumption. In order to shrink toward a low-rank representation, we investigate penalized least squares estimators with a Schatten-p quasi-norm penalty term, p≤1. We study these estimators under two possible assumptions—a modified version of the restricted isometry condition and a uniform bound on the ratio “empirical norm induced by the sampling operator/Frobenius norm.” The main results are stated as nonasymptotic upper bounds on the prediction risk and on the Schatten-q risk of the estimators, where q∈[p, 2]. The rates that we obtain for the prediction risk are of the form rm/N (for m=T), up to logarithmic factors, where r is the rank of A. The particular examples of multi-task learning and matrix completion are worked out in detail. The proofs are based on tools from the theory of empirical processes. As a by-product, we derive bounds for the kth entropy numbers of the quasi-convex Schatten class embeddings SpM↪S2M, p<1, which are of independent interest.

Unified Low-Rank Matrix Estimate Via Penalized Matrix Least Squares Approximation

On the Schatten P-Quasi-norm Minimization for Low-Rank Matrix Recovery

Estimation of high-dimensional low-rank matrices

A Unified Computational and Statistical Framework for Nonconvex Low-rank Matrix Estimation

The Estimate Of The Rank For Regression Coefficient Matrix In A Median Regression-Model

Low-rank matrix estimation in multi-response regression with measurement errors: Statistical and computational guarantees.

Low-rank matrix estimation via nonconvex spectral regularized methods in errors-in-variables matrix regression

Sparse and Low-Rank Matrix Quantile Estimation with Application to Quadratic Regression

Low-Rank Matrix Approximation with Stability.

Projected Gradient Descent Algorithm for Low-Rank Matrix Estimation

Fast low-rank estimation by projected gradient descent: General statistical and algorithmic guarantees

Uncertainty Quantification in Bayesian Reduced-Rank Sparse Regressions

The statistical rate for support matrix machines under low rankness and row (column) sparsity

Robust Regularized Low-Rank Matrix Models for Regression and Classification

Rank-Constrained Least-Squares: Prediction and Inference

Bayesian rank penalization.

The linearized alternating direction method of multipliers for low-rank and fused LASSO matrix regression model

Low-Rank Matrix Estimation from Rank-One Projections by Unlifted Convex Optimization

Smoothed Low Rank and Sparse Matrix Recovery by Iteratively Reweighted Least Squares Minimization

Multivariate Spatiotemporal Models with Low Rank Coefficient Matrix

Bayesian Learning for Low-Rank matrix reconstruction