Abstract:With powerful parallel computing GPUs and massive user data, neural-network-based deep learning can well exert its strong power in problem modeling and solving, and has archived great success in many applications such as image classification, speech recognition and machine translation etc. While deep learning has been increasingly popular, the problem of privacy leakage becomes more and more urgent. Given the fact that the training data may contain highly sensitive information, e.g., personal medical records, directly sharing them among the users (i.e., participants) or centrally storing them in one single location may pose a considerable threat to user privacy. In this paper, we present a practical privacy-preserving collaborative deep learning system that allows users to cooperatively build a collective deep learning model with data of all participants, without direct data sharing and central data storage. In our system, each participant trains a local model with their own data and only shares model parameters with the others. To further avoid potential privacy leakage from sharing model parameters, we use functional mechanism to perturb the objective function of the neural network in the training process to achieve $\epsilon $ -differential privacy. In particular, for the first time, we consider the existence of unreliable participants , i.e., the participants with low-quality data, and propose a solution to reduce the impact of these participants while protecting their privacy. We evaluate the performance of our system on two well-known real-world datasets for regression and classification tasks. The results demonstrate that the proposed system is robust against unreliable participants, and achieves high accuracy close to the model trained in a traditional centralized manner while ensuring rigorous privacy protection.

Collaborative and Privacy-Preserving Machine Teaching via Consensus Optimization

Privacy-Preserving Collaborative Deep Learning with Unreliable Participants.

Robust Federated Training via Collaborative Machine Teaching using Trusted Instances

Iterative Teacher-Aware Learning

Manipulating Predictions over Discrete Inputs in Machine Teaching

Fairness-Driven Private Collaborative Machine Learning

Generalizing Differentially Private Decentralized Deep Learning with Multi-Agent Consensus

Optimal Collusion-Free Teaching

Machine Teaching: A New Paradigm for Building Machine Learning Systems

Robust and privacy-preserving collaborative training: a comprehensive survey

Unlearn What You Have Learned: Adaptive Crowd Teaching with Exponentially Decayed Memory Learners

Collaborative Learning via Prediction Consensus

Teaching the Old Dog New Tricks: Supervised Learning with Constraints

Using Machine Teaching to Investigate Human Assumptions when Teaching Reinforcement Learners

Understanding the Role of Adaptivity in Machine Teaching: The Case of Version Space Learners

Collaborative Active Learning in Conditional Trust Environment

An Optimal Control Approach to Sequential Machine Teaching

Understanding the Power and Limitations of Teaching with Imperfect Knowledge

Representational Alignment Supports Effective Machine Teaching

Machine Teaching of Active Sequential Learners

Learning to Teach in Cooperative Multiagent Reinforcement Learning