Abstract:The increasing Internet-of-Things (IoT) devices have produced large volumes of data. A deep learning technique is widely used to analyze the potential value of these data due to its unprecedented performance in both the academic and industrial communities. However, the data generated from the IoT devices are distributed among different users. Directly combining these data to a central server will cause privacy leakage, especially for personal sensitive data. Rather than centralized training by getting access to all these raw data, an alternative is to collaboratively learn a model in a distributed manner. However, there exist two main challenges in a distributed learning setting. The first one is how to preserve the privacy of users. The second one is to reduce the communication burden (e.g., mobile users have limited bandwidth) due to high-frequent data exchange. To address these two challenges, we design a communication efficient and privacy-preserving framework to enable different participants to distributively learn a model with a privacy protection guarantee. In particular, we develop a differentially private approximate mechanism for the distributed deep learning. In addition, we design a new gradient sparsification method to, at the first time, reduce both upload and download communication costs. The performance of the proposed framework is tested under different neural network structures for different data sets including, image classification and mobile sensor data. The experimental results demonstrate that we can reduce the communication up to only 2% compared to the full gradients exchange and achieve up to 16% accuracy increase compared to the previous works.

Differential Privacy and Distributed Online Learning for Wireless Big Data.

NetDP: In-Network Differential Privacy for Large-Scale Data Processing

Distributed Private Online Learning for Social Big Data Computing over Data Center Networks

Distributed Dynamic Online Learning with Differential Privacy Via Path-Length Measurement.

Wireless Coded Distributed Learning with Gaussian-based Local Differential Privacy.

Locally Differentially Private Distributed Online Learning with Guaranteed Optimality

Approximate to Be Great: Communication Efficient and Privacy-Preserving Large-Scale Distributed Deep Learning in Internet of Things

Privacy-Preserving Distributed Optimization and Learning

Joint Communication and Learning Design of Differential Privacy for Federated Learning over Multi-cell Networks.

Enabling Differentially Private in Big Data Machine Learning

Decentralized Wireless Federated Learning with Differential Privacy

Locally Differentially Private Gradient Tracking for Distributed Online Learning over Directed Graphs

Differential Privacy Preserving in Big Data Analytics for Connected Health

On differential privacy for federated learning in wireless systems with multiple base stations

Differential Privacy-preserving Distributed Machine Learning

Differential Privacy of Big Data: an Overview

Hybrid Differential Privacy Based Federated Learning for Internet of Things

Local Differential Private Spatio- Temporal Dynamic Graph Learning for Wireless Social Networks

A Privacy-preserving Approach to Distributed Set-membership Estimation over Wireless Sensor Networks.

Distributed Logistic Regression with Differential Privacy

Differentially Private Distributed Estimation and Learning