Abstract:With powerful parallel computing GPUs and massive user data, neural-network-based deep learning can well exert its strong power in problem modeling and solving, and has archived great success in many applications such as image classification, speech recognition and machine translation etc. While deep learning has been increasingly popular, the problem of privacy leakage becomes more and more urgent. Given the fact that the training data may contain highly sensitive information, e.g., personal medical records, directly sharing them among the users (i.e., participants) or centrally storing them in one single location may pose a considerable threat to user privacy. In this paper, we present a practical privacy-preserving collaborative deep learning system that allows users to cooperatively build a collective deep learning model with data of all participants, without direct data sharing and central data storage. In our system, each participant trains a local model with their own data and only shares model parameters with the others. To further avoid potential privacy leakage from sharing model parameters, we use functional mechanism to perturb the objective function of the neural network in the training process to achieve $\epsilon $ -differential privacy. In particular, for the first time, we consider the existence of unreliable participants , i.e., the participants with low-quality data, and propose a solution to reduce the impact of these participants while protecting their privacy. We evaluate the performance of our system on two well-known real-world datasets for regression and classification tasks. The results demonstrate that the proposed system is robust against unreliable participants, and achieves high accuracy close to the model trained in a traditional centralized manner while ensuring rigorous privacy protection.

Split Learning with Differential Privacy for Integrated Terrestrial and Non-Terrestrial Networks

Privacy-Preserving Collaborative Deep Learning with Unreliable Participants.

An Efficient Privacy-aware Split Learning Framework for Satellite Communications

Federated Split Learning with Data and Label Privacy Preservation in Vehicular Networks

Secure Split Learning Against Property Inference, Data Reconstruction, and Feature Space Hijacking Attacks

Joint Communication and Learning Design of Differential Privacy for Federated Learning over Multi-cell Networks.

Evaluating Privacy Leakage in Split Learning

Enhancing Accuracy-Privacy Trade-off in Differentially Private Split Learning

Enhancing Privacy in ControlNet and Stable Diffusion via Split Learning

Split Ways: Privacy-Preserving Training of Encrypted Data Using Split Learning

Performance-Enhanced Federated Learning with Differential Privacy for Internet of Things

Privacy-Preserving Split Learning with Vision Transformers using Patch-Wise Random and Noisy CutMix

Advancements of federated learning towards privacy preservation: from federated learning to split learning

PPSFL: Privacy-Preserving Split Federated Learning for heterogeneous data in edge-based Internet of Things

Split Learning without Local Weight Sharing to Enhance Client-side Data Privacy

PSLF: Defending Against Label Leakage in Split Learning

Split-and-Denoise: Protect large language model inference with local differential privacy

Local Differential Privacy Based Membership-Privacy-Preserving Federated Learning for Deep-Learning-Driven Remote Sensing

Split-U-Net: Preventing Data Leakage in Split Learning for Collaborative Multi-Modal Brain Tumor Segmentation

Make Split, not Hijack: Preventing Feature-Space Hijacking Attacks in Split Learning

Hybrid Differential Privacy Based Federated Learning for Internet of Things