Abstract:Estimating the density of a distribution from samples is a fundamental problem in statistics. In many practical settings, the Wasserstein distance is an appropriate error metric for density estimation. For example, when estimating population densities in a geographic region, a small Wasserstein distance means that the estimate is able to capture roughly where the population mass is. In this work we study differentially private density estimation in the Wasserstein distance. We design and analyze instance-optimal algorithms for this problem that can adapt to easy instances. For distributions $P$ over $\mathbb{R}$, we consider a strong notion of instance-optimality: an algorithm that uniformly achieves the instance-optimal estimation rate is competitive with an algorithm that is told that the distribution is either $P$ or $Q_P$ for some distribution $Q_P$ whose probability density function (pdf) is within a factor of 2 of the pdf of $P$. For distributions over $\mathbb{R}^2$, we use a different notion of instance optimality. We say that an algorithm is instance-optimal if it is competitive with an algorithm that is given a constant-factor multiplicative approximation of the density of the distribution. We characterize the instance-optimal estimation rates in both these settings and show that they are uniformly achievable (up to polylogarithmic factors). Our approach for $\mathbb{R}^2$ extends to arbitrary metric spaces as it goes via hierarchically separated trees. As a special case our results lead to instance-optimal private learning in TV distance for discrete distributions.

Optimal Rates for Estimation of Two-Dimensional Totally Positive Distributions

Convolutions of Totally Positive Distributions with applications to Kernel Density Estimation

Instance-Optimal Differentially Private Estimation

Learning High-dimensional Gaussian Graphical Models under Total Positivity Without Adjustment of Tuning Parameters

Instance-Optimal Private Density Estimation in the Wasserstein Distance

Generalized multi-view model: Adaptive density estimation under low-rank constraints

Pathwise Derivatives for Multivariate Distributions

Riemannian radial distributions on Riemannian symmetric spaces: Optimal rates of convergence for parameter estimation

Optimal rates of convergence for estimating the null density and proportion of nonnull effects in large-scale multiple testing

Parametric-Rate Inference for One-Sided Differentiable Parameters

A Few Interactions Improve Distributed Nonparametric Estimation, Optimally

Conditional Optimization of Algorithms for Estimating Distributions of Solutions to Stochastic Differential Equations

Optimal unbiased estimators via convex hulls

Semi-Discrete Optimal Transport: Nearly Minimax Estimation With Stochastic Gradient Descent and Adaptive Entropic Regularization

On Semi-Supervised Estimation of Distributions

Nonparametric Estimation via Partial Derivatives

Beyond Catoni: Sharper Rates for Heavy-Tailed and Robust Mean Estimation

Distribution Estimation under the Infinity Norm

Minimax rates for latent position estimation in the generalized random dot product graph

Statistical Convergence Rates of Optimal Transport Map Estimation between General Distributions

Neural Estimation Of Entropic Optimal Transport