Abstract:This paper provides an alternative to penalized estimators for estimation and vari- able selection in high dimensional linear regression models with measurement error or missing covariates. We propose estimation via bias corrected least squares after model selection. We show that by separating model selection and estimation, it is possible to achieve an improved rate of convergence of the L2 estimation error compared to the rate sqrt{s log p/n} achieved by simultaneous estimation and variable selection methods such as L1 penalized corrected least squares. If the correct model is selected with high probability then the L2 rate of convergence for the proposed method is indeed the oracle rate of sqrt{s/n}. Here s, p are the number of non zero parameters and the model dimension, respectively, and n is the sample size. Under very general model selection criteria, the proposed method is computationally simpler and statistically at least as efficient as the L1 penalized corrected least squares method, performs model selection without the availability of the bias correction matrix, and is able to provide estimates with only a small sub-block of the bias correction covariance matrix of order s x s in comparison to the p x p correction matrix required for computation of the L1 penalized version. Furthermore we show that the model selection requirements are met by a correlation screening type method and the L1 penalized corrected least squares method. Also, the proposed methodology when applied to the estimation of precision matrices with missing observations, is seen to perform at least as well as existing L1 penalty based methods. All results are supported empirically by a simulation study.

Weighted Lasso Estimates for Sparse Logistic Regression: Non-Asymptotic Properties with Measurement Errors

Average case analysis of Lasso under ultra-sparse conditions

Adaptive Lasso in high-dimensional settings

Weighted LASSO for Sparse Recovery with Statistical Prior Support Information

Robust adaptive LASSO in high-dimensional logistic regression

Penalized robust estimators in logistic regression with applications to sparse models

Asymptotic Properties of Lasso+mLS and Lasso+Ridge in Sparse High-Dimensional Linear Regression

Sparse Estimation Via ℓ_q Optimization Method in High-Dimensional Linear Regression

The Lasso under Poisson-Like Heteroscedasticity

Penalized Sparse Covariance Regression with High Dimensional Covariates

Sparse Density Estimation with Measurement Errors

Sparse Estimation Via Lower-Order Penalty Optimization Methods in High-Dimensional Linear Regression.

Robust Fused Lasso Penalized Huber Regression with Nonasymptotic Property and Implementation Studies

Sparse Poisson Regression with Penalized Weighted Score Function

Minimax rates of l p-losses for high-dimensional linear regression models with additive measurement errors over l q-balls

Two Stage Non-penalized Corrected Least Squares for High Dimensional Linear Models with Measurement error or Missing Covariates

High-dimensional Measurement Error Models for Lipschitz Loss

Sparse-penalized deep neural networks estimator under weak dependence

A new penalized least absolute deviation model for high dimensional sparse linear regression and an efficient sequential linear programming algorithm

Oracle Inequalities for the Lasso in the Cox Model

On High dimensional Poisson models with measurement error: hypothesis testing for nonlinear nonconvex optimization