Abstract:In this paper we revisit the classical method of partitioning classification and study its convergence rate under relaxed conditions, both for observable (non-privatised) and for privatised data. Let the feature vector $X$ take values in $\mathbb{R}^d$ and denote its label by $Y$. Previous results on the partitioning classifier worked with the strong density assumption, which is restrictive, as we demonstrate through simple examples. We assume that the distribution of $X$ is a mixture of an absolutely continuous and a discrete distribution, such that the absolutely continuous component is concentrated to a $d_a$ dimensional subspace. Here, we study the problem under much milder assumptions: in addition to the standard Lipschitz and margin conditions, a novel characteristic of the absolutely continuous component is introduced, by which the exact convergence rate of the classification error probability is calculated, both for the binary and for the multi-label cases. Interestingly, this rate of convergence depends only on the intrinsic dimension $d_a$. The privacy constraints mean that the data $(X_1,Y_1), \dots ,(X_n,Y_n)$ cannot be directly observed, and the classifiers are functions of the randomised outcome of a suitable local differential privacy mechanism. The statistician is free to choose the form of this privacy mechanism, and here we add Laplace distributed noises to the discontinuations of all possible locations of the feature vector $X_i$ and to its label $Y_i$. Again, tight upper bounds on the rate of convergence of the classification error probability are derived, without the strong density assumption, such that this rate depends on $2\,d_a$.

Classification under local differential privacy

Locally Differentially Private Naive Bayes Classification

Differentially Private Fair Binary Classifications

Differential Privacy for Sparse Classification Learning

Towards Understanding the Fairness of Differentially Private Margin Classifiers

Minimax Optimal Two-Sample Testing under Local Differential Privacy

Large Margin Multiclass Gaussian Classification with Differential Privacy

Differentially Private Variational Inference for Non-conjugate Models

Learning with User-Level Local Differential Privacy

Instance-Optimal Differentially Private Estimation

Differential Privacy Under Class Imbalance: Methods and Empirical Insights

Statistical Quantification of Differential Privacy: A Local Approach

Optimal Locally Private Nonparametric Classification with Public Data

Geometrizing rates of convergence under local differential privacy constraints

Finite Sample Differentially Private Confidence Intervals

Differentially Private Algorithms for Empirical Machine Learning

Label differential privacy via clustering

On Rate-Optimal Partitioning Classification from Observable and from Privatised Data

Statistical Inference in the Differential Privacy Model

Locally differentially private estimation of nonlinear functionals of discrete distributions

Learning Privately from Multiparty Data