Abstract:We study the problem of robustly estimating the mean or location parameter without moment assumptions. We show that for a large class of symmetric distributions, the same error as in the Gaussian setting can be achieved efficiently. The distributions we study include products of arbitrary symmetric one-dimensional distributions, such as product Cauchy distributions, as well as elliptical distributions. For product distributions and elliptical distributions with known scatter (covariance) matrix, we show that given an $\varepsilon$-corrupted sample, we can with probability at least $1-\delta$ estimate its location up to error $O(\varepsilon \sqrt{\log(1/\varepsilon)})$ using $\tfrac{d\log(d) + \log(1/\delta)}{\varepsilon^2 \log(1/\varepsilon)}$ samples. This result matches the best-known guarantees for the Gaussian distribution and known SQ lower bounds (up to the $\log(d)$ factor). For elliptical distributions with unknown scatter (covariance) matrix, we propose a sequence of efficient algorithms that approaches this optimal error. Specifically, for every $k \in \mathbb{N}$, we design an estimator using time and samples $\tilde{O}({d^k})$ achieving error $O(\varepsilon^{1-\frac{1}{2k}})$. This matches the error and running time guarantees when assuming certifiably bounded moments of order up to $k$. For unknown covariance, such error bounds of $o(\sqrt{\varepsilon})$ are not even known for (general) sub-Gaussian distributions. Our algorithms are based on a generalization of the well-known filtering technique. We show how this machinery can be combined with Huber-loss-based techniques to work with projections of the noise that behave more nicely than the initial noise. Moreover, we show how SoS proofs can be used to obtain algorithmic guarantees even for distributions without a first moment. We believe that this approach may find other applications in future works.

Robust estimations from distribution structures: II. Central Moments

Robust estimations from distribution structures: III. Invariant Moments

Robust estimations from distribution structures: I. Mean

A central limit theorem for studentized U-statistics

Limit theorems for kernel density estimators under dependent samples

Bounded distributions place limits on skewness and larger moments

On the Maximum Likelihood Estimators' Uniqueness and Existence for Two Unitary Distributions: Analytically and Graphically, with Application

Robust Mean Estimation Without Moments for Symmetric Distributions

A NEW PERSPECTIVE ON ROBUST M-ESTIMATION: FINITE SAMPLE THEORY AND APPLICATIONS TO DEPENDENCE-ADJUSTED MULTIPLE TESTING

Weighted skewness and kurtosis unbiased by sample size

Oracle inequalities and upper bounds for kernel conditional U-statistics estimators on manifolds and more general metric spaces associated with operators

A stochastic ordering of multiple hypergeometric laws: Peakedness of category counts about half the population category sizes is symmetric unimodal in the sample size

Estimation Beyond Data Reweighting: Kernel Method of Moments

A High-dimensional Convergence Theorem for U-statistics with Applications to Kernel-based Testing

Mode-based estimation of the center of symmetry

Precise Asymptotics of U-Statistics

Asymptotics for non-degenerate multivariate $U$-statistics with estimated nuisance parameters under the null and local alternative hypotheses

Some properties of the unified skew-normal distribution

Distributionally Robust Chance Constraint with Unimodality-Skewness Information and Conic Reformulation

Generating Multivariate Ordinal Data via Entropy Principles