Abstract:We study the problem of robustly estimating the mean or location parameter without moment assumptions. We show that for a large class of symmetric distributions, the same error as in the Gaussian setting can be achieved efficiently. The distributions we study include products of arbitrary symmetric one-dimensional distributions, such as product Cauchy distributions, as well as elliptical distributions. For product distributions and elliptical distributions with known scatter (covariance) matrix, we show that given an $\varepsilon$-corrupted sample, we can with probability at least $1-\delta$ estimate its location up to error $O(\varepsilon \sqrt{\log(1/\varepsilon)})$ using $\tfrac{d\log(d) + \log(1/\delta)}{\varepsilon^2 \log(1/\varepsilon)}$ samples. This result matches the best-known guarantees for the Gaussian distribution and known SQ lower bounds (up to the $\log(d)$ factor). For elliptical distributions with unknown scatter (covariance) matrix, we propose a sequence of efficient algorithms that approaches this optimal error. Specifically, for every $k \in \mathbb{N}$, we design an estimator using time and samples $\tilde{O}({d^k})$ achieving error $O(\varepsilon^{1-\frac{1}{2k}})$. This matches the error and running time guarantees when assuming certifiably bounded moments of order up to $k$. For unknown covariance, such error bounds of $o(\sqrt{\varepsilon})$ are not even known for (general) sub-Gaussian distributions. Our algorithms are based on a generalization of the well-known filtering technique. We show how this machinery can be combined with Huber-loss-based techniques to work with projections of the noise that behave more nicely than the initial noise. Moreover, we show how SoS proofs can be used to obtain algorithmic guarantees even for distributions without a first moment. We believe that this approach may find other applications in future works.

Robust Estimate with Minimum Mean Squared Error

A Novel Robust Smooth Support Vector Machine

Robust Estimation of Measurement Error Covariance

Compromised imputation based mean estimators using robust quantile regression

Robust Minimax Estimation Based on Particle Swarm Optimization

Weighted least squares regression with the best robustness and high computability

Unified robust estimation

Robust estimations from distribution structures: I. Mean

A New Interpretation on the MMSE As a Robust MEE Criterion

Novel Robust Least-Squares Estimator for Linear Dynamic Data Reconciliation

Robust small area estimation for unit level model with density power divergence

Quasi-weighted least squares estimator for data reconciliation

Universal Robust Regression via Maximum Mean Discrepancy

Robust explicit estimators using the power-weighted repeated medians

Uniform bounds for robust mean estimators

Minimum Sum of Distances Estimator: Robustness and Stability

Robust Mean Estimation Without Moments for Symmetric Distributions

Risk-Aware MMSE Estimation

Robustifying Markowitz

Robust MAVE for single-index varying-coefficient models

Robust Regression over Averaged Uncertainty