Abstract:Learning of matrix-valued data has recently surged in a range of scientific and business applications. Trace regression is a widely used method to model effects of matrix predictors and has shown great success in matrix learning. However, nearly all existing trace regression solutions rely on two assumptions: (i) a known functional form of the conditional mean, and (ii) a global low-rank structure in the entire range of the regression function, both of which may be violated in practice. In this article, we relax these assumptions by developing a general framework for nonparametric trace regression models via structured sign series representations of high dimensional functions. The new model embraces both linear and nonlinear trace effects, and enjoys rank invariance to order-preserving transformations of the response. In the context of matrix completion, our framework leads to a substantially richer model based on what we coin as the "sign rank" of a matrix. We show that the sign series can be statistically characterized by weighted classification tasks. Based on this connection, we propose a learning reduction approach to learn the regression model via a series of classifiers, and develop a parallelable computation algorithm to implement sign series aggregations. We establish the excess risk bounds, estimation error rates, and sample complexities. Our proposal provides a broad nonparametric paradigm to many important matrix learning problems, including matrix regression, matrix completion, multi-task learning, and compressed sensing. We demonstrate the advantages of our method through simulations and two applications, one on brain connectivity study and the other on high-rank image completion.

Generalized High-Dimensional Trace Regression via Nuclear Norm Regularization

Nonparametric Trace Regression in High Dimensions via Sign Series Representation

Matrix completion by Truncated Nuclear Norm Regularization

Fast and Accurate Matrix Completion Via Truncated Nuclear Norm Regularization

Double Weighted Truncated Nuclear Norm Regularization for Low-Rank Matrix Completion.

Partial Trace Regression and Low-Rank Kraus Decomposition

Exact Low Tubal Rank Tensor Recovery from Gaussian Measurements

Optimal spectral norm rates for noisy low-rank matrix completion

Euclidean-Norm-Induced Schatten-p Quasi-Norm Regularization for Low-Rank Tensor Completion and Tensor Robust Principal Component Analysis

Efficient Low-rank Identification via Accelerated Iteratively Reweighted Nuclear Norm Minimization

Near-Optimal differentially private low-rank trace regression with guaranteed private initialization

Safe subspace screening for the adaptive nuclear norm regularized trace regression

Truncated nuclear norm regularization for low-rank tensor completion

Large Scale Multi-Class Classification with Truncated Nuclear Norm Regularization

Tensor Robust Principal Component Analysis with a New Tensor Nuclear Norm

Low-Rank Approximation via Generalized Reweighted Iterative Nuclear and Frobenius Norms

Robust Regularized Low-Rank Matrix Models for Regression and Classification

A robust low-rank matrix completion based on truncated nuclear norm and Lp-norm

Low-Rank Tensor Learning by Generalized Nonconvex Regularization

Learning Linear and Nonlinear Low-Rank Structure in Multi-Task Learning

Low-Rank Tensor Completion by Truncated Nuclear Norm Regularization