Abstract:With powerful parallel computing GPUs and massive user data, neural-network-based deep learning can well exert its strong power in problem modeling and solving, and has archived great success in many applications such as image classification, speech recognition and machine translation etc. While deep learning has been increasingly popular, the problem of privacy leakage becomes more and more urgent. Given the fact that the training data may contain highly sensitive information, e.g., personal medical records, directly sharing them among the users (i.e., participants) or centrally storing them in one single location may pose a considerable threat to user privacy. In this paper, we present a practical privacy-preserving collaborative deep learning system that allows users to cooperatively build a collective deep learning model with data of all participants, without direct data sharing and central data storage. In our system, each participant trains a local model with their own data and only shares model parameters with the others. To further avoid potential privacy leakage from sharing model parameters, we use functional mechanism to perturb the objective function of the neural network in the training process to achieve $\epsilon $ -differential privacy. In particular, for the first time, we consider the existence of unreliable participants , i.e., the participants with low-quality data, and propose a solution to reduce the impact of these participants while protecting their privacy. We evaluate the performance of our system on two well-known real-world datasets for regression and classification tasks. The results demonstrate that the proposed system is robust against unreliable participants, and achieves high accuracy close to the model trained in a traditional centralized manner while ensuring rigorous privacy protection.

SecBERT: Privacy-preserving pre-training based neural network inference system

GELU-Net: A Globally Encrypted, Locally Unencrypted Deep Neural Network for Privacy-Preserved Learning.

Privacy-Preserving Collaborative Deep Learning with Unreliable Participants.

FEDBFPT: an Efficient Federated Learning Framework for BERT Further Pre-Training

Practical Two-party Privacy-preserving Neural Network Based on Secret Sharing

A Distributed Privacy-Preserving Framework for Deep Learning with Edge-Cloud Computing.

Communication-Efficient Privacy-Preserving Neural Network Inference via Arithmetic Secret Sharing

FastSecNet: An Efficient Cryptographic Framework for Private Neural Network Inference

SECO: Secure Inference With Model Splitting Across Multi-Server Hierarchy

SecPE: Secure Prompt Ensembling for Private and Robust Large Language Models

FssNN: Communication-Efficient Secure Neural Network Training via Function Secret Sharing

Training Encrypted Models with Privacy-preserved Data on Blockchain

$S^2$NeRF: Privacy-preserving Training Framework for NeRF

PrivyNet: A Flexible Framework for Privacy-Preserving Deep Neural Network Training

BAYHENN: Combining Bayesian Deep Learning and Homomorphic Encryption for Secure DNN Inference

PCNNCEC: Efficient and Privacy-Preserving Convolutional Neural Network Inference Based on Cloud-Edge-Client Collaboration

Bi-CryptoNets: Leveraging Different-Level Privacy for Encrypted Inference

PriPHiT: Privacy-Preserving Hierarchical Training of Deep Neural Networks

PyHENet: A Generic Framework for Privacy-Preserving DL Inference Based on Fully Homomorphic Encryption

Achieving Efficient and Privacy-Preserving Neural Network Training and Prediction in Cloud Environments

Secure Deep Neural Network Models Publishing Against Membership Inference Attacks Via Training Task Parallelism