Abstract:We present differentially private algorithms for high-dimensional mean estimation. Previous private estimators on distributions over $\mathbb{R}^d$ suffer from a curse of dimensionality, as they require $\Omega(d^{1/2})$ samples to achieve non-trivial error, even in cases where $O(1)$ samples suffice without privacy. This rate is unavoidable when the distribution is isotropic, namely, when the covariance is a multiple of the identity matrix, or when accuracy is measured with respect to the affine-invariant Mahalanobis distance. Yet, real-world data is often highly anisotropic, with signals concentrated on a small number of principal components. We develop estimators that are appropriate for such signals$\unicode{x2013}$our estimators are $(\varepsilon,\delta)$-differentially private and have sample complexity that is dimension-independent for anisotropic subgaussian distributions. Given $n$ samples from a distribution with known covariance-proxy $\Sigma$ and unknown mean $\mu$, we present an estimator $\hat{\mu}$ that achieves error $\|\hat{\mu}-\mu\|_2\leq \alpha$, as long as $n\gtrsim\mathrm{tr}(\Sigma)/\alpha^2+ \mathrm{tr}(\Sigma^{1/2})/(\alpha\varepsilon)$. In particular, when $\pmb{\sigma}^2=(\sigma_1^2, \ldots, \sigma_d^2)$ are the singular values of $\Sigma$, we have $\mathrm{tr}(\Sigma)=\|\pmb{\sigma}\|_2^2$ and $\mathrm{tr}(\Sigma^{1/2})=\|\pmb{\sigma}\|_1$, and hence our bound avoids dimension-dependence when the signal is concentrated in a few principal components. We show that this is the optimal sample complexity for this task up to logarithmic factors. Moreover, for the case of unknown covariance, we present an algorithm whose sample complexity has improved dependence on the dimension, from $d^{1/2}$ to $d^{1/4}$.

A General Framework for Accurate and Private Mean Estimation.

Robust and Differentially Private Mean Estimation

Instance-optimal Mean Estimation Under Differential Privacy

Private Mean Estimation with Person-Level Differential Privacy

Differentially-Private Collaborative Online Personalized Mean Estimation

Dimension-free Private Mean Estimation for Anisotropic Distributions

AAA: an Adaptive Mechanism for Locally Differential Private Mean Estimation

Mean Estimation Under Heterogeneous Privacy Demands

Covariance-Aware Private Mean Estimation Without Private Covariance Estimation

A Huber Loss Minimization Approach to Mean Estimation under User-level Differential Privacy

Mean Estimation Under Heterogeneous Privacy: Some Privacy Can Be Free

Average-Case Averages: Private Algorithms for Smooth Sensitivity and Mean Estimation

PLAN: Variance-Aware Private Mean Estimation

On Differentially Private U Statistics

Differentially private sub-Gaussian location estimators

Efficient Estimation of a Gaussian Mean with Local Differential Privacy

Mean estimation in the add-remove model of differential privacy

The Cost of Privacy: Optimal Rates of Convergence for Parameter Estimation with Differential Privacy

A Bias-Accuracy-Privacy Trilemma for Statistical Estimation

Better and Simpler Lower Bounds for Differentially Private Statistical Estimation