Abstract:Data privacy is of great concern in cloud machine-learning service platforms, when sensitive data are exposed to service providers. While private computing environments (e.g., secure enclaves), and cryptographic approaches (e.g., homomorphic encryption) provide strong privacy protection, their computing performance still falls short compared to cloud GPUs. To achieve privacy protection with high computing performance, we propose Delta, a new private training and inference framework, with comparable model performance as non-private centralized training. Delta features two asymmetric data flows: the main information-sensitive flow and the residual flow. The main part flows into a small model while the residuals are offloaded to a large model. Specifically, Delta embeds the information-sensitive representations into a low-dimensional space while pushing the information-insensitive part into high-dimension residuals. To ensure privacy protection, the low-dimensional information-sensitive part is secured and fed to a small model in a private environment. On the other hand, the residual part is sent to fast cloud GPUs, and processed by a large model. To further enhance privacy and reduce the communication cost, Delta applies a random binary quantization technique along with a DP-based technique to the residuals before sharing them with the public platform. We theoretically show that Delta guarantees differential privacy in the public environment and greatly reduces the complexity in the private environment. We conduct empirical analyses on CIFAR-10, CIFAR-100 and ImageNet datasets and ResNet-18 and ResNet-34, showing that Delta achieves strong privacy protection, fast training, and inference without significantly compromising the model utility.

Fine-Grained Delta Privacy Preservation for Hierarchical Contexts

A Distributed Privacy-Preserving Framework for Deep Learning with Edge-Cloud Computing.

Integrating Differential Privacy and Contextual Integrity

C-fDRL: Context-Aware Privacy-Preserving Offloading through Federated Deep Reinforcement Learning in Cloud-Enabled IoT

Protecting Activity Sensing Data Privacy Using Hierarchical Information Dissociation

Distilling at the Edge: A Local Differential Privacy Obfuscation Framework for IoT Data Analytics

A Differentially Private Framework for the Dynamic Heterogeneous Redundant Architecture System in Cyberspace

Hierarchical fuzzy neural networks with privacy preservation for heterogeneous big data

All Rivers Run to the Sea: Private Learning with Asymmetric Flows

Hierarchical federated learning with global differential privacy

Fuzzy Privacy Decision for Context-Aware Access Personal Information

Real-time and private spatio-temporal data aggregation with local differential privacy

Extensive Form Game Analysis Based on Context Privacy Preservation for Smart Phone Applications.

Trusted Decentralized Access Control for Client-Context Privacy Preservation

Re-Evaluating Privacy in Centralized and Decentralized Learning: An Information-Theoretical and Empirical Study

A Learning-based Declarative Privacy-Preserving Framework for Federated Data Management

Differentially private data release through multidimensional partitioning

Gradient-Level Differential Privacy Against Attribute Inference Attack for Speech Emotion Recognition

DASS: A Web-Based Fine-Grained Data Access System for Smartphones

FORESEEN: Towards Differentially Private Deep Inference for Intelligent Internet of Things

A two‐stage privacy protection mechanism based on blockchain in mobile crowdsourcing