Abstract:Logistic regression is key method for modeling the probability of a binary outcome based on a collection of covariates. However, the classical formulation of logistic regression relies on the independent sampling assumption, which is often violated when the outcomes interact through an underlying network structure, such as over a temporal/spatial domain or on a social network. This necessitates the development of models that can simultaneously handle both the network `peer-effect' and the effect of high-dimensional covariates. In this paper, we develop a framework for incorporating such dependencies in a high-dimensional logistic regression model by introducing a quadratic interaction term, as in the Ising model, designed to capture the pairwise interactions from the underlying network. The resulting model can also be viewed as an Ising model, where the node-dependent external fields linearly encode the high-dimensional covariates. We propose a penalized maximum pseudo-likelihood method for estimating the network peer-effect and the effect of the covariates (the regression coefficients), which, in addition to handling the high-dimensionality of the parameters, conveniently avoids the computational intractability of the maximum likelihood approach. Under various standard regularity conditions, we show that the corresponding estimate attains the classical high-dimensional rate of consistency. Our results imply that even under network dependence it is possible to consistently estimate the model parameters at the same rate as in classical (independent) logistic regression, when the true parameter is sparse and the underlying network is not too dense. We also develop an efficient algorithm for computing the estimates and validate our theoretical results in numerical experiments. An application to selecting genes in clustering spatial transcriptomics data is also discussed.

High-dimensional classification by sparse logistic regression

On high-dimensional classification by sparse generalized Bayesian logistic regression

Minimax sparse logistic regression for very high-dimensional feature selection.

Robust adaptive LASSO in high-dimensional logistic regression

Penalized robust estimators in logistic regression with applications to sparse models

High‐dimensional sparse classification using exponential weighting with empirical hinge loss

Sparse logistic functional principal component analysis for binary data

Efficient and robust high-dimensional sparse logistic regression via nonlinear primal-dual hybrid gradient algorithms

Classification by sparse generalized additive models

Fully Bayesian logistic regression with hyper-LASSO priors for high-dimensional feature selection

On Regularized Sparse Logistic Regression

Feature Screening Strategy for Non-Convex Sparse Logistic Regression with Log Sum Penalty

High Dimensional Classification with combined Adaptive Sparse PLS and Logistic Regression

Local logistic regression for dimension reduction in classification

Simultaneous Dimension Reduction and Variable Selection for Multinomial Logistic Regression

High-dimensional logistic entropy clustering

High-dimensional regression with a count response

High Dimensional Logistic Regression Under Network Dependence

A Naïve Bayes Regularized Logistic Regression Estimator for Low-dimensional Classification

Prediction and Variable Selection in High-Dimensional Misspecified Binary Classification

A Safe Screening Rule for Sparse Logistic Regression