Abstract:This paper introduces a new regularized version of the robust $\tau$-regression estimator for analyzing high-dimensional datasets subject to gross contamination in the response variables and covariates (explanatory variables). The resulting estimator, termed adaptive $\tau$-Lasso, is robust to outliers and high-leverage points. It also incorporates an adaptive $\ell_1$-norm penalty term, which enables the selection of relevant variables and reduces the bias associated with large true regression coefficients. More specifically, this adaptive $\ell_1$-norm penalty term assigns a weight to each regression coefficient. For a fixed number of predictors $p$, we show that the adaptive $\tau$-Lasso has the oracle property, ensuring both variable-selection consistency and asymptotic normality. Asymptotic normality applies only to the entries of the regression vector corresponding to the true support, assuming knowledge of the true regression vector support. We characterize its robustness by establishing the finite-sample breakdown point and the influence function. We carry out extensive simulations and observe that the class of $\tau$-Lasso estimators exhibits robustness and reliable performance in both contaminated and uncontaminated data settings. We also validate our theoretical findings on robustness properties through simulations. In the face of outliers and high-leverage points, the adaptive $\tau$-Lasso and $\tau$-Lasso estimators achieve the best performance or close-to-best performance in terms of prediction and variable selection accuracy compared to other competing regularized estimators for all scenarios considered in this study. Therefore, the adaptive $\tau$-Lasso and $\tau$-Lasso estimators provide attractive tools for a variety of sparse linear regression problems, particularly in high-dimensional settings and when the data is contaminated by outliers and high-leverage points.

A Tuning-free Robust and Efficient Approach to High-dimensional Regression

Adaptive Lasso in high-dimensional settings

High-dimensional regression in practice: an empirical study of finite-sample prediction, variable selection and ranking

One-Step Regularized Estimator for High-Dimensional Regression Models

A Constructive Approach to High-dimensional Regression

Statistical inference for high dimensional regression via Constrained Lasso

Consistent Tuning Parameter Selection in High Dimensional Sparse Linear Regression

Nonnegative Adaptive Lasso for Ultra-High Dimensional Regression Models and A Two-Stage Method Applied in Financial Modeling

Efficient smoothed concomitant lasso estimation for high dimensional regression

New Robust Variable Selection Methods for Linear Regression Models

Estimation of Error Variance in Regularized Regression Models Via Adaptive Lasso

Asymptotic Properties of Lasso+mLS and Lasso+Ridge in Sparse High-Dimensional Linear Regression

A New Principle for Tuning-Free Huber Regression

Computationally Efficient and Statistically Optimal Robust High-Dimensional Linear Regression

The Adaptive $τ$-Lasso: Robustness and Oracle Properties

Efficient Inference on High-Dimensional Linear Models with Missing Outcomes

High dimensional data regression using Lasso model and neural networks with random weights.

Regression Coefficient and Autoregressive Order Shrinkage and Selection Via the Lasso

Robust adaptive LASSO in high-dimensional logistic regression

New Error Analysis for Lasso

Enhancing efficiency and robustness in high-dimensional linear regression with additional unlabeled data