Abstract:We consider the recovery of regression coefficients, denoted by $\boldsymbol{\beta}_0$, for a single index model (SIM) relating a binary outcome $Y$ to a set of possibly high dimensional covariates $\boldsymbol{X}$, based on a large but 'unlabeled' dataset $\mathcal{U}$, with $Y$ never observed. On $\mathcal{U}$, we fully observe $\boldsymbol{X}$ and additionally, a surrogate $S$ which, while not being strongly predictive of $Y$ throughout the entirety of its support, can forecast it with high accuracy when it assumes extreme values. Such datasets arise naturally in modern studies involving large databases such as electronic medical records (EMR) where $Y$, unlike $(\boldsymbol{X}, S)$, is difficult and/or expensive to obtain. In EMR studies, an example of $Y$ and $S$ would be the true disease phenotype and the count of the associated diagnostic codes respectively. Assuming another SIM for $S$ given $\boldsymbol{X}$, we show that under sparsity assumptions, we can recover $\boldsymbol{\beta}_0$ proportionally by simply fitting a least squares LASSO estimator to the subset of the observed data on $(\boldsymbol{X}, S)$ restricted to the extreme sets of $S$, with $Y$ imputed using the surrogacy of $S$. We obtain sharp finite sample performance bounds for our estimator, including deterministic deviation bounds and probabilistic guarantees. We demonstrate the effectiveness of our approach through multiple simulation studies, as well as by application to real data from an EMR study conducted at the Partners HealthCare Systems.

Stable Direction Recovery in Single-Index Models with a Diverging Number of Predictors

Nonconcave Penalized Inverse Regression in Single-Index Models with High Dimensional Predictors

A distribution-based LASSO for a general single-index model

Variable Selection in a Class of Single-Index Models

Adaptive post-Dantzig estimation and prediction for non-sparse "large p and small n" models

Adjusting Inverse Regression for Predictors with Clustered Distribution

Robust Inference for High‐dimensional Single Index Models

Direction Estimation in the Single-Index Model with Missing Values

Penalized least squares estimation with weakly dependent data

Signed Support Recovery for Single Index Models in High-Dimensions

On Distribution-Weighted Partial Least Squares with Diverging Number of Highly Correlated Predictors

Estimation and Adaptive-to-model Testing for Regressions with Diverging Number of Predictors

Dimension reduction and predictor selection in semiparametric models

Sparsity Identification for High-Dimensional Partially Linear Model with Measurement Error.

Simultaneous support recovery in high dimensions: Benefits and perils of block $\ell_1/\ell_\infty$-regularization

Adaptive Elastic-Net for General Single-Index Regression Models.

New Efficient Estimation and Variable Selection in Models with Single-Index Structure

Adaptive Confidence Region for the Direction in Semiparametric Regressions.

Surrogate Aided Unsupervised Recovery of Sparse Signals in Single Index Models for Binary Outcomes

On the Optimality of Sliced Inverse Regression in High Dimensions

NONCONCAVE PENALIZED M-ESTIMATION WITH A DIVERGING NUMBER OF PARAMETERS