Abstract:In this paper, we study high-dimensional multivariate logistic regression models in which a common set of covariates is used to predict multiple binary outcomes simultaneously. Our work is primarily motivated from many biomedical studies with correlated multiple responses such as the cancer cell-line encyclopedia project. We assume that the underlying regression coefficient matrix is simultaneously low-rank and row-wise sparse. We propose an intuitively appealing selection and estimation framework based on marginal model likelihood, and we develop an efficient computational algorithm for inference. We establish a novel high-dimensional theory for this nonlinear multivariate regression. Our theory is general, allowing for potential correlations between the binary responses. We propose a new type of nuclear norm penalty using the smooth clipped absolute deviation, filling the gap in the related non-convex penalization literature. We theoretically demonstrate that the proposed approach improves estimation accuracy by considering multiple responses jointly through the proposed estimator when the underlying coefficient matrix is low-rank and row-wise sparse. In particular, we establish the non-asymptotic error bounds, and both rank and row support consistency of the proposed method. Moreover, we develop a consistent rule to simultaneously select the rank and row dimension of the coefficient matrix. Furthermore, we extend the proposed methods and theory to a joint Ising model, which accounts for the dependence relationships. In our analysis of both simulated data and the cancer cell line encyclopedia data, the proposed methods outperform the existing methods in better predicting responses.

L2RM: Low-Rank Linear Regression Models for High-Dimensional Matrix Responses

A framework of regularized low-rank matrix models for regression and classification

Low-rank regression models for multiple binary responses and their applications to cancer cell-line encyclopedia data

Adaptive estimation of the rank of the coefficient matrix in high dimensional multivariate response regression models

Robust Regularized Low-Rank Matrix Models for Regression and Classification

Bayesian Longitudinal Low-Rank Regression Models for Imaging Genetic Data from Longitudinal Studies.

Regularized Matrix Regression

Low-rank matrix estimation in multi-response regression with measurement errors: Statistical and computational guarantees.

Linear Hypothesis Testing in Linear Models With High-Dimensional Responses

Testing a Single Regression Coefficient in High Dimensional Linear Models

Low-rank matrix estimation via nonconvex spectral regularized methods in errors-in-variables matrix regression

Low-rank matrix estimation via nonconvex optimization methods in multi-response errors-in-variables regression

Testing a Single Regression Coefficient in High Dimensional Regression Model

Variable Selection in High-Dimensional Sparse Multiresponse Linear Regression Models

Large-Scale Sparse Regression for Multiple Responses with Applications to UK Biobank

On The Equivalent Of Low-Rank Regressions And Linear Discriminant Analysis Based Regressions

Novel Method of Improved Low Rank Linear Regression

RObust rank correlation based screening

Quantized Low-Rank Multivariate Regression with Random Dithering

The linearized alternating direction method of multipliers for low-rank and fused LASSO matrix regression model

Interaction screening in high-dimensional multi-response regression via projected distance correlation