Abstract:The two-sided matrix regression model $Y = A^*X B^* +E$ aims at predicting $Y$ by taking into account both linear links between column features of $X$, via the unknown matrix $B^*$, and also among the row features of $X$, via the matrix $A^*$. We propose low-rank predictors in this high-dimensional matrix regression model via rank-penalized and nuclear norm-penalized least squares. Both criteria are non jointly convex; however, we propose explicit predictors based on SVD and show optimal prediction bounds. We give sufficient conditions for consistent rank selector. We also propose a fully data-driven rank-adaptive procedure. Simulation results confirm the good prediction and the rank-consistency results under data-driven explicit choices of the tuning parameters and the scaling parameter of the noise.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to predict the target matrix $Y$ through the Two - sided Matrix Regression (2MR) model in the context of high - dimensional data. Specifically, the model $Y = A^*XB^*+E$ aims to predict the matrix $Y$ by considering the linear relationships between the column features of $X$ (through the unknown matrix $B^*$) and the linear relationships between the row features of $X$ (through the matrix $A^*$). In the paper, low - rank predictors are proposed and these predictors are implemented by rank - penalty and nuclear - norm - penalty least - squares methods. Although the proposed criteria are not jointly convex, the authors propose explicit predictors based on Singular Value Decomposition (SVD) and show the optimal prediction bounds. In addition, the paper also provides sufficient conditions to ensure the consistency of the rank selector and proposes a fully data - driven rank - adaptive procedure. ### Key Points Summary: 1. **Model Background**: The paper discusses how to use the two - sided matrix regression model to handle high - dimensional data in the context of big data, especially when the data matrix $Y$ can be explained by the design matrix $X$ and two parameter matrices $A^*$ and $B^*$. 2. **Model Form**: The form of the two - sided matrix regression model is $Y = A^*XB^* + E$, where $E$ is the noise matrix and its entries are assumed to be independent, centered $\sigma$-sub - Gaussian distributions. 3. **Low - Rank Predictors**: The paper proposes a method to construct low - rank predictors by rank - penalty and nuclear - norm - penalty least - squares methods. Although these criteria are not jointly convex, the authors give explicit predictors through Singular Value Decomposition (SVD). 4. **Theoretical Results**: The authors prove that these predictors can reach the optimal prediction bounds under certain conditions and provide a data - driven rank - adaptive procedure. 5. **Numerical Simulation**: The numerical simulation results verify the good prediction performance and rank consistency of the proposed predictors under different noise levels. ### Main Contributions: - Propose low - rank predictors and implement these predictors by rank - penalty and nuclear - norm - penalty least - squares methods. - Demonstrate the excellent performance of these predictors in high - dimensional data and provide theoretical guarantees. - Propose a data - driven rank - adaptive procedure that can effectively select the appropriate rank in practical applications. ### Formula Summary: - **Model**: \[ Y = A^* X B^* + E \] - **Low - Rank Predictors**: \[ \hat{A}_r = U_Y \hat{A}_0^r U_X^T \] and \[ \hat{B}_r = V_X \hat{B}_0^r V_Y^T \] - **Nuclear - Norm - Penalty Least - Squares**: \[ \min_{A, B} \| Y - A X B \|_F^2+ 2\lambda\cdot\| A X B \|_* \] Through these methods, the paper provides an effective solution set for the matrix regression problem of high - dimensional data.

Two-sided Matrix Regression

Robust Regularized Low-Rank Matrix Models for Regression and Classification

Multiple Regression for Matrix and Vector Predictors: Models, Theory, Algorithms, and Beyond

Regularized Matrix Regression

Projection-pursuit Bayesian regression for symmetric matrix predictors

Regression-aware decompositions

Estimation of high-dimensional low-rank matrices

The statistical rate for support matrix machines under low rankness and row (column) sparsity

Low-rank matrix estimation via nonconvex spectral regularized methods in errors-in-variables matrix regression

The Estimate Of The Rank For Regression Coefficient Matrix In A Median Regression-Model

Rank-k 2-D Multinomial Logistic Regression for Matrix Data Classification.

L2RM: Low-Rank Linear Regression Models for High-Dimensional Matrix Responses

Nearest Neighbors for Matrix Estimation Interpreted as Blind Regression for Latent Variable Model

Dimension free ridge regression

Predictive Low Rank Matrix Learning under Partial Observations: Mixed-Projection ADMM

Efficient Nonconvex Optimization for Two-way Sparse Reduced-Rank Regression

Unified Low-Rank Matrix Estimate Via Penalized Matrix Least Squares Approximation

Outlier-robust sparse/low-rank least-squares regression and robust matrix completion

Regression for matrix-valued data via Kronecker products factorization

On a Biased Prediction Based on Optimal Mean Square Error Criterion

On adaptivity and minimax optimality of two-sided nearest neighbors