Abstract:We consider observations $(X,y)$ from single index models with unknown link function, Gaussian covariates and a regularized M-estimator $\hat\beta$ constructed from convex loss function and regularizer. In the regime where sample size $n$ and dimension $p$ are both increasing such that $p/n$ has a finite limit, the behavior of the empirical distribution of $\hat\beta$ and the predicted values $X\hat\beta$ has been previously characterized in a number of models: The empirical distributions are known to converge to proximal operators of the loss and penalty in a related Gaussian sequence model, which captures the interplay between ratio $p/n$, loss, regularization and the data generating process. This connection between$(\hat\beta,X\hat\beta)$ and the corresponding proximal operators require solving fixed-point equations that typically involve unobservable quantities such as the prior distribution on the index or the link function. This paper develops a different theory to describe the empirical distribution of $\hat\beta$ and $X\hat\beta$: Approximations of $(\hat\beta,X\hat\beta)$ in terms of proximal operators are provided that only involve observable adjustments. These proposed observable adjustments are data-driven, e.g., do not require prior knowledge of the index or the link function. These new adjustments yield confidence intervals for individual components of the index, as well as estimators of the correlation of $\hat\beta$ with the index. The interplay between loss, regularization and the model is thus captured in a data-driven manner, without solving the fixed-point equations studied in previous works. The results apply to both strongly convex regularizers and unregularized M-estimation. Simulations are provided for the square and logistic loss in single index models including logistic regression and 1-bit compressed sensing with 20\% corrupted bits.

Oracally Efficient Estimation for Single-Index Link Function with Simultaneous Confidence Band

Oracle-efficient M-estimation for single-index models with a smooth simultaneous confidence band

Empirical Likelihood for Single-Index Regression Models under Negatively Associated Errors

Oracally Efficient Estimation of Autoregressive Error Distribution with Simultaneous Confidence Band

Simultaneous semiparametric inference for single-index models

Simultaneous confidence bands for nonparametric regression with partially missing covariates

FUSED KERNEL-SPLINE SMOOTHING FOR REPEATEDLY MEASURED OUTCOMES IN A GENERALIZED PARTIALLY LINEAR MODEL WITH FUNCTIONAL SINGLE INDEX

Robust confidence regions for the index and functional coefficients in the single-index varying coefficients regression model

A smooth simultaneous confidence band for correlation curve

Estimation and convergence rates in the distributional single index model

High-Dimensional Single-Index Models: Link Estimation and Marginal Inference

Empirical likelihood for single-index varying-coefficient models

Functional linear and single-index models: A unified approach via Gaussian Stein identity

Efficient estimation in semivarying coefficient models for longitudinal/clustered data

Doubly robust and efficient estimators for heteroscedastic partially linear single-index models allowing high dimensional covariates

On Single-Index Coefficient Regression Models

Efficient Estimation in Single Index Models through Smoothing splines

Estimating the Conditional Single-Index Error Distribution with A Partial Linear Mean Regression

Observable adjustments in single-index models for regularized M-estimators

Simultaneous Confidence Band for Partially Linear Panel Data Models with Fixed Effects

ASYMPTOTIC DISTRIBUTIONS FOR TWO ESTIMATORS OF THE SINGLE-INDEX MODEL