Abstract:We consider training machine learning models using Training data located on multiple private and geographically-scattered servers with different privacy settings. Due to the distributed nature of the data, communicating with all collaborating private data owners simultaneously may prove challenging or altogether impossible. In this paper, we develop differentially-private asynchronous algorithms for collaboratively training machine-learning models on multiple private datasets. The asynchronous nature of the algorithms implies that a central learner interacts with the private data owners one-on-one whenever they are available for communication without needing to aggregate query responses to construct gradients of the entire fitness function. Therefore, the algorithm efficiently scales to many data owners. We define the cost of privacy as the difference between the fitness of a privacy-preserving machine-learning model and the fitness of trained machine-learning model in the absence of privacy concerns. We prove that we can forecast the performance of the proposed privacy-preserving asynchronous algorithms. We demonstrate that the cost of privacy has an upper bound that is inversely proportional to the combined size of the training datasets squared and the sum of the privacy budgets squared. We validate the theoretical results with experiments on financial and medical datasets. The experiments illustrate that collaboration among more than 10 data owners with at least 10,000 records with privacy budgets greater than or equal to 1 results in a superior machine-learning model in comparison to a model trained in isolation on only one of the datasets, illustrating the value of collaboration and the cost of the privacy. The number of the collaborating datasets can be lowered if the privacy budget is higher.

Towards Privacy-Preserving Model Selection

Privacy Preserving Distributed DBSCAN Clustering

Privacy Preserving PCA for Multiparty Modeling

On the Computational Complexity of Private High-dimensional Model Selection

Privacy Preserving Feature Selection in Distributed Environment

Scalable Differentially Private Model Publishing Via Private Iterative Sample Selection

A privacy-preserving model based on differential approach for sensitive data in cloud environment

Providing Accurate Models across Private Partitioned Data: Secure Maximum Likelihood Estimation

Cross-model Fairness: Empirical Study of Fairness and Ethics Under Model Multiplicity

Privacy-Preserving Generalized Linear Models using Distributed Block Coordinate Descent

Privacy-preserving Data Selection for Horizontal and Vertical Federated Learning

Privacy-preserving two-parties logistic regression on vertically partitioned data using asynchronous gradient sharing

Secure SVM Training over Vertically-Partitioned Datasets Using Consortium Blockchain for Vehicular Social Networks

Privacy Preserving Analytics on Distributed Medical Data

Differential Privacy with Multiple Selections

Distributed Modelling Approaches for Data Privacy Preserving

Personalization Improves Privacy-Accuracy Tradeoffs in Federated Learning

Privacy Preserving Vertical Federated Learning for Tree-based Models

Privacy-Accuracy Trade-Off in Distributed Data Mining

The Cost of Privacy in Asynchronous Differentially-Private Machine Learning

Privacy-Accuracy Trade-Off in Differentially-Private Distributed Classification: A Game Theoretical Approach