Abstract:In collaborative deep learning, current learning frameworks follow either a centralized architecture or a distributed architecture. Whilst centralized architecture deploys a central server to train a global model over the massive amount of joint data from all parties, distributed architecture aggregates parameter updates from participating parties' local model training, via a parameter server. These two server-based architectures present security and robustness vulnerabilities such as single-point-of-failure, single-point-of-breach, privacy leakage, and lack of fairness. To address these problems, we design, implement, and evaluate a purely decentralized privacy-preserving deep learning framework, called DPPDL. DPPDL makes the first investigation on the research problem of fairness in collaborative deep learning, and simultaneously provides fairness and privacy by proposing two novel algorithms: initial benchmarking and privacy-preserving collaborative deep learning. During initial benchmarking, each party trains a local Differentially Private Generative Adversarial Network (DPGAN) and publishes the generated privacy-preserving artificial samples for other parties to label, based on the quality of which to initialize local credibility list for other parties. The local credibility list reflects how much one party contributes to another party, and it is used and updated during collaborative learning to ensure fairness. To protect gradients transaction during privacy-preserving collaborative deep learning, we further put forward a three-layer onion-style encryption scheme. We experimentally demonstrate, on benchmark image datasets, that accuracy, privacy and fairness in collaborative deep learning can be effectively addressed at the same time by our proposed DPPDL framework. Moreover, DPPDL provides a viable solution to detect and isolate the cheating party in the system.

Rethinking Privacy Preserving Deep Learning: How to Evaluate and Thwart Privacy Attacks

Privacy-Preserving Collaborative Deep Learning with Unreliable Participants.

A Distributed Privacy-Preserving Framework for Deep Learning with Edge-Cloud Computing.

A comprehensive survey and taxonomy on privacy-preserving deep learning

Privacy-Preserving Deep Learning on Machine Learning as a Service—a Comprehensive Survey

Deep Learning with Data Privacy via Residual Perturbation

Privacy-Preserving Deep Learning with SPDZ

When Deep Learning Meets Differential Privacy: Privacy,Security, and More

Distributed Layer-Partitioned Training for Privacy-Preserved Deep Learning

Privacy Threats and Protection in Machine Learning

Privacy-Preserving Machine Learning: Methods, Challenges and Directions

Towards Fair and Decentralized Privacy-Preserving Deep Learning with Blockchain.

DPAdapter: Improving Differentially Private Deep Learning through Noise Tolerance Pre-training

Privacy Analysis of Deep Learning in the Wild: Membership Inference Attacks against Transfer Learning

Evaluating Privacy Leakage in Split Learning

A Method to Facilitate Membership Inference Attacks in Deep Learning Models

The Communication-Friendly Privacy-Preserving Machine Learning against Malicious Adversaries

Protection Against Reconstruction and Its Applications in Private Federated Learning

Federated Deep Learning with Bayesian Privacy

Privacy for Rescue: A New Testimony Why Privacy is Vulnerable In Deep Models

On the Evaluation of User Privacy in Deep Neural Networks using Timing Side Channel