Abstract:We present a Distributionally Robust Optimization (DRO) approach to estimate a robustified regression plane in a linear regression setting, when the observed samples are potentially contaminated with adversarially corrupted outliers. Our approach mitigates the impact of outliers by hedging against a family of probability distributions on the observed data, some of which assign very low probabilities to the outliers. The set of distributions under consideration are close to the empirical distribution in the sense of the Wasserstein metric. We show that this DRO formulation can be relaxed to a convex optimization problem which encompasses a class of models. By selecting proper norm spaces for the Wasserstein metric, we are able to recover several commonly used regularized regression models. We provide new insights into the regularization term and give guidance on the selection of the regularization coefficient from the standpoint of a confidence region. We establish two types of performance guarantees for the solution to our formulation under mild conditions. One is related to its out-of-sample behavior (prediction bias), and the other concerns the discrepancy between the estimated and true regression planes (estimation bias). Extensive numerical results demonstrate the superiority of our approach to a host of regression models, in terms of the prediction and estimation accuracies. We also consider the application of our robust learning procedure to outlier detection, and show that our approach achieves a much higher AUC (Area Under the ROC Curve) than M-estimation (Huber, 1964, 1973).

Efficient and Robust Estimation of Linear Regression with Normal Errors

Efficient and robust estimation for autoregressive regression models using shape mixtures of skewt normal distribution

Robust inference for linear regression models with possibly skewed error distribution

Bayesian Robustness to Outliers in Linear Regression and Ratio Estimation

A robust alternative to the Lilliefors test of normality

A NEW PERSPECTIVE ON ROBUST M-ESTIMATION: FINITE SAMPLE THEORY AND APPLICATIONS TO DEPENDENCE-ADJUSTED MULTIPLE TESTING

A Robust Adaptive Modified Maximum Likelihood Estimator for the Linear Regression Model

Robust regression based on shrinkage estimators

A redescending M-estimator approach for outlier-resilient modeling

Adaptive Linear Estimating Equations

A Robust Learning Approach for Regression Models Based on Distributionally Robust Optimization.

Log-regularly varying scale mixture of normals for robust regression

COMPARISONS BETWEEN ROBUST REGRESSION APPROACHES IN THE PRESENCE OF OUTLIERS AND HIGH LEVERAGE POINTS

Correlation-Based Robust Linear Regression with Iterative Outlier Removal

Consistency and Normality Ofm-Estimators in Partly Linear Models with Stochastic Adapted Errors

Sparse and Robust Linear Regression: An Optimization Algorithm and Its Statistical Properties

Online and Offline Robust Multivariate Linear Regression

DETECTION OF MULTIPLE OUTLIERS BY RANDOM ROBUST TESTING

A Robust Learning Algorithm for Regression Models Using Distributionally Robust Optimization under the Wasserstein Metric

Robust Estimation Using Modified Huber's Functions with New Tails.

Robustifying a (local, nonparametric) regression estimator