Abstract:With powerful parallel computing GPUs and massive user data, neural-network-based deep learning can well exert its strong power in problem modeling and solving, and has archived great success in many applications such as image classification, speech recognition and machine translation etc. While deep learning has been increasingly popular, the problem of privacy leakage becomes more and more urgent. Given the fact that the training data may contain highly sensitive information, e.g., personal medical records, directly sharing them among the users (i.e., participants) or centrally storing them in one single location may pose a considerable threat to user privacy. In this paper, we present a practical privacy-preserving collaborative deep learning system that allows users to cooperatively build a collective deep learning model with data of all participants, without direct data sharing and central data storage. In our system, each participant trains a local model with their own data and only shares model parameters with the others. To further avoid potential privacy leakage from sharing model parameters, we use functional mechanism to perturb the objective function of the neural network in the training process to achieve $\epsilon $ -differential privacy. In particular, for the first time, we consider the existence of unreliable participants , i.e., the participants with low-quality data, and propose a solution to reduce the impact of these participants while protecting their privacy. We evaluate the performance of our system on two well-known real-world datasets for regression and classification tasks. The results demonstrate that the proposed system is robust against unreliable participants, and achieves high accuracy close to the model trained in a traditional centralized manner while ensuring rigorous privacy protection.

Collaborative Learning for Less Online Retraining of Neural Receivers

Privacy-Preserving Collaborative Deep Learning with Unreliable Participants.

Over-the-air Learning Rate Optimization for Federated Learning

Asynchronous Federated Learning over Wireless Communication Networks.

Intra-Model Collaborative Learning of Neural Networks.

Efficient Collaborative Learning over Unreliable D2D Network: Adaptive Cluster Head Selection and Resource Allocation

Importance-Aware Data Selection and Resource Allocation in Federated Edge Learning System.

Neural Network-Assisted Receiver Design Via Learning Trellis Diagram Online

A Privacy-preserving Distributed Training Framework for Cooperative Multi-agent Deep Reinforcement Learning

Asynchronous Online Adaptation via Modular Drift Detection for Deep Receivers

Online Knowledge Distillation via Collaborative Learning

A Novel Neural Network Training Framework with Data Assimilation

Adaptive Personalized Over-the-Air Federated Learning with Reflecting Intelligent Surfaces

OLNC: Online Learning of Network Conditions for Adaptive Video Streaming.

Symbol-Level Online Channel Tracking for Deep Receivers

AC-DNN: An Adaptive Compact DNNs Architecture for Collaborative Learning Among Heterogeneous Smart Devices

Trust-Region Adaptive Frequency for Online Continual Learning

Online/Offline Learning to Enable Robust Beamforming: Limited Feedback Meets Deep Generative Models

Adaptive Model Pruning and Personalization for Federated Learning Over Wireless Networks

Learning to Collaborate in Decentralized Learning of Personalized Models

Pre-Training and Personalized Fine-Tuning via Over-the-Air Federated Meta-Learning: Convergence-Generalization Trade-Offs