Abstract:This paper provides an alternative to penalized estimators for estimation and vari- able selection in high dimensional linear regression models with measurement error or missing covariates. We propose estimation via bias corrected least squares after model selection. We show that by separating model selection and estimation, it is possible to achieve an improved rate of convergence of the L2 estimation error compared to the rate sqrt{s log p/n} achieved by simultaneous estimation and variable selection methods such as L1 penalized corrected least squares. If the correct model is selected with high probability then the L2 rate of convergence for the proposed method is indeed the oracle rate of sqrt{s/n}. Here s, p are the number of non zero parameters and the model dimension, respectively, and n is the sample size. Under very general model selection criteria, the proposed method is computationally simpler and statistically at least as efficient as the L1 penalized corrected least squares method, performs model selection without the availability of the bias correction matrix, and is able to provide estimates with only a small sub-block of the bias correction covariance matrix of order s x s in comparison to the p x p correction matrix required for computation of the L1 penalized version. Furthermore we show that the model selection requirements are met by a correlation screening type method and the L1 penalized corrected least squares method. Also, the proposed methodology when applied to the estimation of precision matrices with missing observations, is seen to perform at least as well as existing L1 penalty based methods. All results are supported empirically by a simulation study.

Adjusting for High-Dimensional Covariates in Sparse Precision Matrix Estimation by Lasso Penalized D-Trace Loss

Confidence Intervals for Sparse Precision Matrix Estimation Via Lasso Penalized D-trace Loss

Adaptive Lasso in high-dimensional settings

Positive-Definite Sparse Precision Matrix Estimation

Efficient Distributed Estimation of High-Dimensional Sparse Precision Matrix

A fast ADMM algorithm for sparse precision matrix estimation using lasso penalized D-trace loss

Penalized Sparse Covariance Regression with High Dimensional Covariates

Distributed Sparse Precision Matrix Estimation via Alternating Block-Based Gradient Descent

Weighted Lasso Estimates for Sparse Logistic Regression: Non-Asymptotic Properties with Measurement Errors

Two Stage Non-penalized Corrected Least Squares for High Dimensional Linear Models with Measurement error or Missing Covariates

Asymptotic Properties of Lasso+mLS and Lasso+Ridge in Sparse High-Dimensional Linear Regression

Sparse estimation of a covariance matrix

Differential Network Analysis via the Lasso Penalized D-Trace Loss

Sparse Estimation Via ℓ_q Optimization Method in High-Dimensional Linear Regression

Robust high-dimensional precision matrix estimation

Lasso with Latents: Efficient Estimation, Covariate Rescaling, and Computational-Statistical Gaps

Differential Network Analysis Via Lasso Penalized D-trace Loss

A Sparse-Group Lasso

A new penalized least absolute deviation model for high dimensional sparse linear regression and an efficient sequential linear programming algorithm

Fused LASSO Penalized Least Absolute Deviation Estimator for High Dimensional Linear Regression

Optimal Designs in Sparse Linear Models