Abstract:In collaborative deep learning, current learning frameworks follow either a centralized architecture or a distributed architecture. Whilst centralized architecture deploys a central server to train a global model over the massive amount of joint data from all parties, distributed architecture aggregates parameter updates from participating parties' local model training, via a parameter server. These two server-based architectures present security and robustness vulnerabilities such as single-point-of-failure, single-point-of-breach, privacy leakage, and lack of fairness. To address these problems, we design, implement, and evaluate a purely decentralized privacy-preserving deep learning framework, called DPPDL. DPPDL makes the first investigation on the research problem of fairness in collaborative deep learning, and simultaneously provides fairness and privacy by proposing two novel algorithms: initial benchmarking and privacy-preserving collaborative deep learning. During initial benchmarking, each party trains a local Differentially Private Generative Adversarial Network (DPGAN) and publishes the generated privacy-preserving artificial samples for other parties to label, based on the quality of which to initialize local credibility list for other parties. The local credibility list reflects how much one party contributes to another party, and it is used and updated during collaborative learning to ensure fairness. To protect gradients transaction during privacy-preserving collaborative deep learning, we further put forward a three-layer onion-style encryption scheme. We experimentally demonstrate, on benchmark image datasets, that accuracy, privacy and fairness in collaborative deep learning can be effectively addressed at the same time by our proposed DPPDL framework. Moreover, DPPDL provides a viable solution to detect and isolate the cheating party in the system.

A Privacy-Preserving and Trustable Multi-agent Learning Framework

Privacy-Preserving Collaborative Deep Learning with Unreliable Participants.

A Distributed Privacy-Preserving Framework for Deep Learning with Edge-Cloud Computing.

A Distributed Trust Framework for Privacy-Preserving Machine Learning

Trusted AI in Multiagent Systems: An Overview of Privacy and Security for Distributed Learning

Generalizing Differentially Private Decentralized Deep Learning with Multi-Agent Consensus

Privacy Protection Technology of Maritime Multi-agent Communication Based on Part-Federated Learning

Privacy-Preserving Federated Learning Framework Based on Chained Secure Multiparty Computing

DeTrust-FL: Privacy-Preserving Federated Learning in Decentralized Trust Setting

Towards Fair and Decentralized Privacy-Preserving Deep Learning with Blockchain.

Privacy preserving distributed machine learning with federated learning

A Privacy-preserving Distributed Training Framework for Cooperative Multi-agent Deep Reinforcement Learning

Secure and Trusted Collaborative Learning Based on Blockchain for Artificial Intelligence of Things

A Blockchain-Based Trustworthy Federated Learning Privacy Protection Scheme

Decentralized Collaborative Learning Framework with External Privacy Leakage Analysis

Insuring against the perils in distributed learning: privacy-preserving empirical risk minimization

Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy

Distributed Modelling Approaches for Data Privacy Preserving

Privacy Preserving Multi-Agent Reinforcement Learning in Supply Chains

Multi-Party Dual Learning

TDML -- A Trustworthy Distributed Machine Learning Framework