Abstract:We consider non-parametric density estimation in the framework of local approximate differential privacy. In contrast to centralized privacy scenarios with a trusted curator, in the local setup anonymization must be guaranteed already on the individual data owners' side and therefore must precede any data mining tasks. Thus, the published anonymized data should be compatible with as many statistical procedures as possible. We suggest adding Laplace noise and Gaussian processes (both appropriately scaled) to kernel density estimators to obtain approximate differential private versions of the latter ones. We obtain minimax type results over Sobolev classes indexed by a smoothness parameter $s>1/2$ for the mean squared error at a fixed point. In particular, we show that taking the average of private kernel density estimators from $n$ different data owners attains the optimal rate of convergence if the bandwidth parameter is correctly specified. Notably, the optimal convergence rate in terms of the sample size $n$ is $n^{-(2s-1)/(2s+1)}$ under local differential privacy and thus deteriorated to the rate $n^{-(2s-1)/(2s)}$ which holds without privacy restrictions. Since the optimal choice of the bandwidth parameter depends on the smoothness $s$ and is thus not accessible in practice, adaptive methods for bandwidth selection are necessary and must, in the local privacy framework, be performed directly on the anonymized data. We address this problem by means of a variant of Lepski's method tailored to the privacy setup and obtain general oracle inequalities for private kernel density estimators. In the Sobolev case, the resulting adaptive estimator attains the optimal rate of convergence at least up to extra logarithmic factors.

Online Distribution Learning with Local Private Constraints

Locally differentially private estimation of nonlinear functionals of discrete distributions

Locally Differentially Private Distributed Online Learning with Guaranteed Optimality

Locally Private Gaussian Estimation

The Limits of Differential Privacy in Online Learning

Learning with User-Level Local Differential Privacy

Privacy-Preserving Distributed Online Optimization Over Unbalanced Digraphs via Subgradient Rescaling

Distribution-Aware Mean Estimation under User-level Local Differential Privacy

Geometrizing rates of convergence under local differential privacy constraints

Near-Optimal Algorithms for Differentially Private Online Learning in a Stochastic Environment

Mutual Information Optimally Local Private Discrete Distribution Estimation

Local Differential Private Data Aggregation for Discrete Distribution Estimation

Compressive Sensing Approaches for Sparse Distribution Estimation Under Local Privacy

Oracle-Efficient Hybrid Online Learning with Unknown Distribution

Differentially Private Distributed Estimation and Learning

Optimal Federated Learning for Nonparametric Regression with Heterogeneous Distributed Differential Privacy Constraints

Collect at Once, Use Effectively: Making Non-interactive Locally Private Learning Possible

Adapting to Online Label Shift with Provable Guarantees

Pointwise adaptive kernel density estimation under local approximate differential privacy

Joint Distribution Analysis for Set-Valued Data With Local Differential Privacy

Distributed, communication-efficient, and differentially private estimation of KL divergence