Abstract:We study the problem of exact support recovery for high-dimensional sparse linear regression under independent Gaussian design when the signals are weak, rare, and possibly heterogeneous. Under a suitable scaling of the sample size and signal sparsity, we fix the minimum signal magnitude at the information-theoretic optimal rate and investigate the asymptotic selection accuracy of best subset selection (BSS) and marginal screening (MS) procedures. We show that despite the ideal setup, somewhat surprisingly, marginal screening can fail to achieve exact recovery with probability converging to one in the presence of heterogeneous signals, whereas BSS enjoys model consistency whenever the minimum signal strength is above the information-theoretic threshold. To mitigate the computational intractability of BSS, we also propose an efficient two-stage algorithmic framework called ETS (Estimate Then Screen) comprised of an estimation step and gradient coordinate screening step, and under the same scaling assumption on sample size and sparsity, we show that ETS achieves model consistency under the same information-theoretic optimal requirement on the minimum signal strength as BSS. Finally, we present a simulation study comparing ETS with LASSO and marginal screening. The numerical results agree with our asymptotic theory even for realistic values of the sample size, dimension and sparsity.

Asymptotic Properties of Marginal Least-Square Estimator for Ultrahigh-Dimensional Linear Regression Models with Correlated Errors

Robust Estimation and Shrinkage in Ultrahigh Dimensional Expectile Regression with Heavy Tails and Variance Heterogeneity

Asymptotic Properties of the Least Squares Estimator in Local to Unity Processes with Fractional Gaussian Noise

Asymptotic Theory of Least Distances Estimate in Multivariate Linear Models

On the Asymptotics of Marginal Regression Splines with Longitudinal Data

Asymptotics of Linear Regression with Linearly Dependent Data

High-dimensional variable selection with heterogeneous signals: A precise asymptotic perspective

Inference for High-Dimensional Linear Expectile Regression with De-Biasing Method

Benign Overfitting of Non-Sparse High-Dimensional Linear Regression with Correlated Noise

Advances in Bayesian model selection consistency for high-dimensional generalized linear models

Dyson Equation for Correlated Linearizations and Test Error of Random Features Regression

Estimation of the asymptotic variance of univariate and multivariate random fields and statistical inference

On the minimax robustness against correlation and heteroscedasticity of ordinary least squares among generalized least squares estimators of regression

Matrix Dyson equation for correlated linearizations and test error of random features regression

Marginal Hazards Model for Multivariate Failure Time Data with Auxiliary Covariates

Marginal expected shortfall inference under multivariate regular variation

A note on marginal correlation based screening

On Extreme Value Asymptotics of Projected Sample Covariances in High Dimensions with Applications in Finance and Convolutional Networks

Asymptotics of estimators for structured covariance matrices

Estimating correlations among elliptically distributed random variables under any form of heteroskedasticity

With random regressors, least squares inference is robust to correlated errors with unknown correlation structure