Abstract:With powerful parallel computing GPUs and massive user data, neural-network-based deep learning can well exert its strong power in problem modeling and solving, and has archived great success in many applications such as image classification, speech recognition and machine translation etc. While deep learning has been increasingly popular, the problem of privacy leakage becomes more and more urgent. Given the fact that the training data may contain highly sensitive information, e.g., personal medical records, directly sharing them among the users (i.e., participants) or centrally storing them in one single location may pose a considerable threat to user privacy. In this paper, we present a practical privacy-preserving collaborative deep learning system that allows users to cooperatively build a collective deep learning model with data of all participants, without direct data sharing and central data storage. In our system, each participant trains a local model with their own data and only shares model parameters with the others. To further avoid potential privacy leakage from sharing model parameters, we use functional mechanism to perturb the objective function of the neural network in the training process to achieve $\epsilon $ -differential privacy. In particular, for the first time, we consider the existence of unreliable participants , i.e., the participants with low-quality data, and propose a solution to reduce the impact of these participants while protecting their privacy. We evaluate the performance of our system on two well-known real-world datasets for regression and classification tasks. The results demonstrate that the proposed system is robust against unreliable participants, and achieves high accuracy close to the model trained in a traditional centralized manner while ensuring rigorous privacy protection.

Privacy-Preserving Vertical Collaborative Logistic Regression without Trusted Third-Party Coordinator

Privacy-Preserving Collaborative Model Learning: the Case of Word Vector Training

Privacy-Preserving Collaborative Deep Learning with Unreliable Participants.

Peer-to-peer privacy-preserving vertical federated learning without trusted third-party coordinator

Privacy-Preserving Vertical Federated Logistic Regression without Trusted Third-Party Coordinator

When Homomorphic Encryption Marries Secret Sharing: Secure Large-Scale Sparse Logistic Regression and Applications in Risk Control.

Efficient Privacy Preserving Logistic Regression for Horizontally Distributed Data

Privacy-preserving two-parties logistic regression on vertically partitioned data using asynchronous gradient sharing

VFLR: An Efficient and Privacy-Preserving Vertical Federated Framework for Logistic Regression

VPPLR: Privacy-preserving logistic regression on vertically partitioned data using vectorization sharing

PEVLR: A New Privacy-Preserving and Efficient Approach for Vertical Logistic Regression.

Privacy-preserving cloud-edge collaborative learning without trusted third-party coordinator

Privacy-preserving multi-party logistic regression in cloud computing

EFMVFL: An Efficient and Flexible Multi-party Vertical Federated Learning without a Third Party

A flexible and privacy-preserving federated learning framework based on logistic regression

PPCL: Privacy-preserving collaborative learning for mitigating indirect information leakage

FedV: Privacy-Preserving Federated Learning over Vertically Partitioned Data

Privacy-Preserving Multiparty Learning For Logistic Regression

Privacy-Preserving Generalized Linear Models using Distributed Block Coordinate Descent

A Study of Secure Algorithms for Vertical Federated Learning: Take Secure Logistic Regression as an Example

Privacy-Preserving Distributed Linear Regression on High-Dimensional Data