Abstract:Algorithmic robust statistics has traditionally focused on the contamination model where a small fraction of the samples are arbitrarily corrupted. We consider a recent contamination model that combines two kinds of corruptions: (i) small fraction of arbitrary outliers, as in classical robust statistics, and (ii) local perturbations, where samples may undergo bounded shifts on average. While each noise model is well understood individually, the combined contamination model poses new algorithmic challenges, with only partial results known. Existing efficient algorithms are limited in two ways: (i) they work only for a weak notion of local perturbations, and (ii) they obtain suboptimal error for isotropic subgaussian distributions (among others). The latter limitation led [NGS24, COLT'24] to hypothesize that improving the error might, in fact, be computationally hard. Perhaps surprisingly, we show that information theoretically optimal error can indeed be achieved in polynomial time, under an even \emph{stronger} local perturbation model (the sliced-Wasserstein metric as opposed to the Wasserstein metric). Notably, our analysis reveals that the entire family of stability-based robust mean estimators continues to work optimally in a black-box manner for the combined contamination model. This generalization is particularly useful in real-world scenarios where the specific form of data corruption is not known in advance. We also present efficient algorithms for distribution learning and principal component analysis in the combined contamination model.

A universal robustification procedure

A NEW PERSPECTIVE ON ROBUST M-ESTIMATION: FINITE SAMPLE THEORY AND APPLICATIONS TO DEPENDENCE-ADJUSTED MULTIPLE TESTING

Asymptotic equivalence and adaptive estimation for robust nonparametric regression

Uniform bounds for robust mean estimators

M robustified additive nonparametric regression

High-dimensional robust regression under heavy-tailed data: Asymptotics and Universality

Optimal Robust Estimation under Local and Global Corruptions: Stronger Adversary and Smaller Error

Universal Robust Regression via Maximum Mean Discrepancy

Asymptotic Theory for Doubly Robust Estimators with Continuous-Time Nuisance Parameters

Statistical Barriers to Affine-equivariant Estimation

Non-Asymptotic Guarantees for Robust Statistical Learning under Infinite Variance Assumption

Bayesian Robustness to Outliers in Linear Regression and Ratio Estimation

A robust alternative to the Lilliefors test of normality

Universal generalization guarantees for Wasserstein distributionally robust models

On Some Asymptotic Properties and an Almost Sure Approximation of the Normalized Inverse-Gaussian Process

A Geometric Perspective on Double Robustness by Semiparametric Theory and Information Geometry

A Geometric Unification of Distributionally Robust Covariance Estimators: Shrinking the Spectrum by Inflating the Ambiguity Set

Law-invariant risk measures: Extension properties and qualitative robustness

Large parameter asymptotic analysis for homogeneous normalized random measures with independent increments

Robust Estimators in High Dimensions without the Computational Intractability

A multivariate normal approximation for the Dirichlet density and some applications