Abstract:This paper introduces a new regularized version of the robust $\tau$-regression estimator for analyzing high-dimensional datasets subject to gross contamination in the response variables and covariates (explanatory variables). The resulting estimator, termed adaptive $\tau$-Lasso, is robust to outliers and high-leverage points. It also incorporates an adaptive $\ell_1$-norm penalty term, which enables the selection of relevant variables and reduces the bias associated with large true regression coefficients. More specifically, this adaptive $\ell_1$-norm penalty term assigns a weight to each regression coefficient. For a fixed number of predictors $p$, we show that the adaptive $\tau$-Lasso has the oracle property, ensuring both variable-selection consistency and asymptotic normality. Asymptotic normality applies only to the entries of the regression vector corresponding to the true support, assuming knowledge of the true regression vector support. We characterize its robustness by establishing the finite-sample breakdown point and the influence function. We carry out extensive simulations and observe that the class of $\tau$-Lasso estimators exhibits robustness and reliable performance in both contaminated and uncontaminated data settings. We also validate our theoretical findings on robustness properties through simulations. In the face of outliers and high-leverage points, the adaptive $\tau$-Lasso and $\tau$-Lasso estimators achieve the best performance or close-to-best performance in terms of prediction and variable selection accuracy compared to other competing regularized estimators for all scenarios considered in this study. Therefore, the adaptive $\tau$-Lasso and $\tau$-Lasso estimators provide attractive tools for a variety of sparse linear regression problems, particularly in high-dimensional settings and when the data is contaminated by outliers and high-leverage points.

Manifold-Regularized Adaptive Lasso.

Adaptive Lasso in high-dimensional settings

Discriminative Lasso

Extended Adaptive Lasso for Multi-Class and Multi-Label Feature Selection.

Robust sparse manifold discriminant analysis

Adaptive Neighborhood Selection for Manifold Learning

Adaptive Manifold Learning

High-order covariate interacted Lasso for feature selection.

Manifold adversarial training for supervised and semi-supervised learning

The Adaptive Lasso and Its Oracle Properties

Manifold Elastic Net: a Unified Framework for Sparse Dimension Reduction

Manifold Regularized Discriminative Feature Selection for Multi-Label Learning

Unsupervised Feature Selection Via Latent Representation Learning and Manifold Regularization

Robust Adaptive Linear Discriminant Analysis With Bidirectional Reconstruction Constraint

Adaptive Lasso, Transfer Lasso, and Beyond: An Asymptotic Perspective

Enhanced Manifold Regularization for Semi-Supervised Classification.

Adaptive nonlinear auto-associative modeling through manifold learning

Adaptive Lasso for High Dimensional Regression and Gaussian Graphical Modeling

Discriminative Sparse Subspace Learning with Manifold Regularization

An Inexact Manifold Augmented Lagrangian Method for Adaptive Sparse Canonical Correlation Analysis with Trace Lasso Regularization

The Adaptive $τ$-Lasso: Robustness and Oracle Properties