Abstract:This paper introduces a new regularized version of the robust $\tau$-regression estimator for analyzing high-dimensional datasets subject to gross contamination in the response variables and covariates (explanatory variables). The resulting estimator, termed adaptive $\tau$-Lasso, is robust to outliers and high-leverage points. It also incorporates an adaptive $\ell_1$-norm penalty term, which enables the selection of relevant variables and reduces the bias associated with large true regression coefficients. More specifically, this adaptive $\ell_1$-norm penalty term assigns a weight to each regression coefficient. For a fixed number of predictors $p$, we show that the adaptive $\tau$-Lasso has the oracle property, ensuring both variable-selection consistency and asymptotic normality. Asymptotic normality applies only to the entries of the regression vector corresponding to the true support, assuming knowledge of the true regression vector support. We characterize its robustness by establishing the finite-sample breakdown point and the influence function. We carry out extensive simulations and observe that the class of $\tau$-Lasso estimators exhibits robustness and reliable performance in both contaminated and uncontaminated data settings. We also validate our theoretical findings on robustness properties through simulations. In the face of outliers and high-leverage points, the adaptive $\tau$-Lasso and $\tau$-Lasso estimators achieve the best performance or close-to-best performance in terms of prediction and variable selection accuracy compared to other competing regularized estimators for all scenarios considered in this study. Therefore, the adaptive $\tau$-Lasso and $\tau$-Lasso estimators provide attractive tools for a variety of sparse linear regression problems, particularly in high-dimensional settings and when the data is contaminated by outliers and high-leverage points.

Choosing the $p$ in $L_p$ loss: rate adaptivity on the symmetric location problem

Robust Estimation and Shrinkage in Ultrahigh Dimensional Expectile Regression with Heavy Tails and Variance Heterogeneity

Expectile regression for analyzing heteroscedasticity in high dimension

Adaptive estimation in symmetric location model under log-concavity constraint

Adaptive $$l_p$$ $$(0<p<1)$$ Regularization: Oracle Property and Applications

Sharp Rates in Dependent Learning Theory: Avoiding Sample Size Deflation for the Square Loss

Minimax rates of convergence for nonparametric location-scale models

A Note on Adaptive Lp Regularization

Robust Regression With Asymmetric Loss Functions

Adaptive estimation over anisotropic functional classes via oracle approach

Noisy recovery from random linear observations: Sharp minimax rates under elliptical constraints

Accuracy Assessment for High-dimensional Linear Regression

Estimating conditional hazard functions and densities with the highly-adaptive lasso

Optimal Excess Risk Bounds for Empirical Risk Minimization on $p$-Norm Linear Regression

Estimation in the convolution structure density model. Part II: adaptation over the scale of anisotropic classes

Minimax rates of l p-losses for high-dimensional linear regression models with additive measurement errors over l q-balls

Local convergence rates of the nonparametric least squares estimator with applications to transfer learning

Adaptive and non-adaptive minimax rates for weighted Laplacian-eigenmap based nonparametric regression

The Adaptive $τ$-Lasso: Robustness and Oracle Properties

Generalization of l1 constraints for high dimensional regression problems

Non-convex Penalized Estimation in High-Dimensional Models with Single-Index Structure