Abstract:We present a Distributionally Robust Optimization (DRO) approach to estimate a robustified regression plane in a linear regression setting, when the observed samples are potentially contaminated with adversarially corrupted outliers. Our approach mitigates the impact of outliers by hedging against a family of probability distributions on the observed data, some of which assign very low probabilities to the outliers. The set of distributions under consideration are close to the empirical distribution in the sense of the Wasserstein metric. We show that this DRO formulation can be relaxed to a convex optimization problem which encompasses a class of models. By selecting proper norm spaces for the Wasserstein metric, we are able to recover several commonly used regularized regression models. We provide new insights into the regularization term and give guidance on the selection of the regularization coefficient from the standpoint of a confidence region. We establish two types of performance guarantees for the solution to our formulation under mild conditions. One is related to its out-of-sample behavior (prediction bias), and the other concerns the discrepancy between the estimated and true regression planes (estimation bias). Extensive numerical results demonstrate the superiority of our approach to a host of regression models, in terms of the prediction and estimation accuracies. We also consider the application of our robust learning procedure to outlier detection, and show that our approach achieves a much higher AUC (Area Under the ROC Curve) than M-estimation (Huber, 1964, 1973).

Robust Dawoud–Kibria estimator for handling multicollinearity and outliers in the linear regression model

Robust Liu Estimator Used to Combat Some Challenges in Partially Linear Regression Model by Improving LTS Algorithm Using Semidefinite Programming

Weighted penalized m-estimators in robust ridge regression: an application to gasoline consumption data

Novel Robust Least-Squares Estimator for Linear Dynamic Data Reconciliation

A Robust Adaptive Modified Maximum Likelihood Estimator for the Linear Regression Model

DETECTION OF MULTIPLE OUTLIERS BY RANDOM ROBUST TESTING

A redescending M-estimator approach for outlier-resilient modeling

Robust Negative Binomial Regression via the Kibria–Lukman Strategy: Methodology and Application

Correlation-Based Robust Linear Regression with Iterative Outlier Removal

A Robust Learning Algorithm for Regression Models Using Distributionally Robust Optimization under the Wasserstein Metric

Outlier detection and robust variable selection via the penalized weighted LAD-LASSO method

Robust Partial Least Squares Using Low Rank and Sparse Decomposition

A Robust Learning Approach for Regression Models Based on Distributionally Robust Optimization.

Some New Robust Estimators for Circular Logistic Regression Model with Applications on Meteorological and Ecological Data

A new ridge type estimator and its performance for the linear regression model: Simulation and application

Robust Iterative Algorithm of Weighted Least Squares Support Vector Machine and Its Application in Spectral Analysis

Penalized Weighted Least Squares for Outlier Detection and Robust Regression

COMPARISONS BETWEEN ROBUST REGRESSION APPROACHES IN THE PRESENCE OF OUTLIERS AND HIGH LEVERAGE POINTS

Robust Estimation of Models for Longitudinal Data with Dropouts and Outliers

Outlier detection using distributionally robust optimization under the Wasserstein metric

A Distributionally Robust Optimization Approach for Outlier Detection *