Abstract:Suppose that we observe entries or, more generally, linear combinations of entries of an unknown m×T-matrix A corrupted by noise. We are particularly interested in the high-dimensional setting where the number mT of unknown entries can be much larger than the sample size N. Motivated by several applications, we consider estimation of matrix A under the assumption that it has small rank. This can be viewed as dimension reduction or sparsity assumption. In order to shrink toward a low-rank representation, we investigate penalized least squares estimators with a Schatten-p quasi-norm penalty term, p≤1. We study these estimators under two possible assumptions—a modified version of the restricted isometry condition and a uniform bound on the ratio “empirical norm induced by the sampling operator/Frobenius norm.” The main results are stated as nonasymptotic upper bounds on the prediction risk and on the Schatten-q risk of the estimators, where q∈[p, 2]. The rates that we obtain for the prediction risk are of the form rm/N (for m=T), up to logarithmic factors, where r is the rank of A. The particular examples of multi-task learning and matrix completion are worked out in detail. The proofs are based on tools from the theory of empirical processes. As a by-product, we derive bounds for the kth entropy numbers of the quasi-convex Schatten class embeddings SpM↪S2M, p<1, which are of independent interest.

The Estimate Of The Rank For Regression Coefficient Matrix In A Median Regression-Model

Unified Low-Rank Matrix Estimate Via Penalized Matrix Least Squares Approximation

Asymptotic Theory of Least Distances Estimate in Multivariate Linear Models

Small-sample Properties for Generalized Ridge Estimator of Regression Coefficients in Linear Regression Model

Robust Regularized Low-Rank Matrix Models for Regression and Classification

Sparse and Low-Rank Matrix Quantile Estimation with Application to Quadratic Regression

Estimation of high-dimensional low-rank matrices

Rank-Constrained Least-Squares: Prediction and Inference

Low-rank matrix estimation via nonconvex spectral regularized methods in errors-in-variables matrix regression

Multivariate Spatiotemporal Models with Low Rank Coefficient Matrix

Low-rank matrix estimation in multi-response regression with measurement errors: Statistical and computational guarantees.

A Minimax Lower Bound for Low-Rank Matrix-Variate Logistic Regression

The linearized alternating direction method of multipliers for low-rank and fused LASSO matrix regression model

The statistical rate for support matrix machines under low rankness and row (column) sparsity

Uniform Post Selection Inference for LAD Regression and Other Z-estimation problems

THE METHOD OF RANK-BASED ROBUST REGRESSION ANALYSIS

Efficient Estimation for Rank-Based Regression with Clustered Data

Generalized multi-view model: Adaptive density estimation under low-rank constraints

Projected Gradient Descent Algorithm for Low-Rank Matrix Estimation

Low-rank matrix estimation via nonconvex optimization methods in multi-response errors-in-variables regression

Causal Interpretation of Regressions With Ranks