Abstract:Designing privacy-preserving machine learning algorithms has received great attention in recent years, especially in the setting when the data contains sensitive information. Differential privacy (DP) is a widely used mechanism for data analysis with privacy guarantees. In this paper, we produce a differentially private random feature model. Random features, which were proposed to approximate large-scale kernel machines, have been used to study privacy-preserving kernel machines as well. We consider the over-parametrized regime (more features than samples) where the non-private random feature model is learned via solving the min-norm interpolation problem, and then we apply output perturbation techniques to produce a private model. We show that our method preserves privacy and derive a generalization error bound for the method. To the best of our knowledge, we are the first to consider privacy-preserving random feature models in the over-parametrized regime and provide theoretical guarantees. We empirically compare our method with other privacy-preserving learning methods in the literature as well. Our results show that our approach is superior to the other methods in terms of generalization performance on synthetic data and benchmark data sets. Additionally, it was recently observed that DP mechanisms may exhibit and exacerbate disparate impact, which means that the outcomes of DP learning algorithms vary significantly among different groups. We show that both theoretically and empirically, random features have the potential to reduce disparate impact, and hence achieve better fairness.

A two-phase random forest with differential privacy

A Differential Privacy Budget Allocation Algorithm Based on Out-of-Bag Estimation in Random Forest

Multinomial random forest

Multinomial Random Forest: Toward Consistency and Privacy-Preservation

Random forest with differential privacy in federated learning framework for network attack detection and classification

Differentially- and non-differentially-private random decision trees

Differentially Private Greedy Decision Forest

On the Gini-impurity Preservation for Privacy Random Forests

Nikolai N. Anichkov and his theory of atherosclerosis.

On Learning Cluster Coefficient of Private Networks

Decision Tree Classification with Differential Privacy: A Survey

A Statistical Framework for Differential Privacy

Verifiable Privacy-preserving Scheme based on Vertical Federated Random Forest

Federated Transfer Learning with Differential Privacy

An Effective Federated Recommendation Framework with Differential Privacy

Differential Private Stack Generalization with an Application to Diabetes Prediction

Decision Making with Differential Privacy under a Fairness Lens

An Adaptive Differential Privacy Method Based on Federated Learning

[Intrauterine contraception in a "double uterus"].

Differential Privacy Hierarchical Federated Learning Method based on Privacy Budget Allocation

Differentially Private Random Feature Model