Abstract:We present a Distributionally Robust Optimization (DRO) approach to estimate a robustified regression plane in a linear regression setting, when the observed samples are potentially contaminated with adversarially corrupted outliers. Our approach mitigates the impact of outliers by hedging against a family of probability distributions on the observed data, some of which assign very low probabilities to the outliers. The set of distributions under consideration are close to the empirical distribution in the sense of the Wasserstein metric. We show that this DRO formulation can be relaxed to a convex optimization problem which encompasses a class of models. By selecting proper norm spaces for the Wasserstein metric, we are able to recover several commonly used regularized regression models. We provide new insights into the regularization term and give guidance on the selection of the regularization coefficient from the standpoint of a confidence region. We establish two types of performance guarantees for the solution to our formulation under mild conditions. One is related to its out-of-sample behavior (prediction bias), and the other concerns the discrepancy between the estimated and true regression planes (estimation bias). Extensive numerical results demonstrate the superiority of our approach to a host of regression models, in terms of the prediction and estimation accuracies. We also consider the application of our robust learning procedure to outlier detection, and show that our approach achieves a much higher AUC (Area Under the ROC Curve) than M-estimation (Huber, 1964, 1973).

Robust explicit estimation of the log-logistic distribution with applications

Robust estimators for the log-logistic model based on ranked set sampling

Estimating the failure rate of the log-logistic distribution by smooth adaptive and bias-correction methods

Robust parameter estimation of the log-logistic distribution based on density power divergence estimators

Robust explicit estimators using the power-weighted repeated medians

Modified generalized method of moments for a robust estimation of polytomous logistic model

Unit Log-Logistic Distribution and Unit Log-Logistic Regression Model

A NEW PERSPECTIVE ON ROBUST M-ESTIMATION: FINITE SAMPLE THEORY AND APPLICATIONS TO DEPENDENCE-ADJUSTED MULTIPLE TESTING

An Optimized Logistic Regression Model Based on the Maximum Entropy Estimation under the Hesitant Fuzzy Environment.

Robust beta regression through the logit transformation

Approximate Maximum Likelihood Estimations for Lognormal Distribution

Robust and sparse estimation methods for high dimensional linear and logistic regression

Robust estimation and bias-corrected empirical likelihood in generalized linear models with right censored data

Log-logistic Distribution as a Reliability Model: A Bayesian Analysis

On the correspondence of deviances and maximum likelihood and interval estimates from log-linear to logistic regression modelling

Distributionally Robust Logistic Regression

A Robust Learning Approach for Regression Models Based on Distributionally Robust Optimization.

Some New Robust Estimators for Circular Logistic Regression Model with Applications on Meteorological and Ecological Data

New robust statistical procedures for polytomous logistic regression models

Uncertain Maximum Likelihood Estimation with Application to Uncertain Regression Analysis

Robust Estimation Using Modified Huber's Functions with New Tails.