Abstract:Federated learning (FL) aims to protect data privacy by cooperatively learning a model without sharing private data among users. For Federated Learning of Deep Neural Network with billions of model parameters, existing privacy-preserving solutions are unsatisfactory. Homomorphic encryption (HE) based methods provide secure privacy protections but suffer from extremely high computational and communication overheads rendering it almost useless in practice . Deep learning with Differential Privacy (DP) was implemented as a practical learning algorithm at a manageable cost in complexity. However, DP is vulnerable to aggressive Bayesian restoration attacks as disclosed in the literature and demonstrated in experimental results of this work. To address the aforementioned perplexity, we propose a novel Bayesian Privacy (BP) framework which enables Bayesian restoration attacks to be formulated as the probability of reconstructing private data from observed public information. Specifically, the proposed BP framework accurately quantifies privacy loss by Kullback-Leibler (KL) Divergence between the prior distribution about the privacy data and the posterior distribution of restoration private data conditioning on exposed information}. To our best knowledge, this Bayesian Privacy analysis is the first to provides theoretical justification of secure privacy-preserving capabilities against Bayesian restoration attacks. As a concrete use case, we demonstrate that a novel federated deep learning method using private passport layers is able to simultaneously achieve high model performance, privacy-preserving capability and low computational complexity. Theoretical analysis is in accordance with empirical measurements of information leakage extensively experimented with a variety of DNN networks on image classification MNIST, CIFAR10, and CIFAR100 datasets.

Privacy-Preserving Deep Learning Models for Law Big Data Feature Learning

Privacy-Preserving Collaborative Deep Learning with Unreliable Participants.

A Distributed Privacy-Preserving Framework for Deep Learning with Edge-Cloud Computing.

Research on the Application of Government Big Data Platform Based on Federated Learning

A comprehensive survey and taxonomy on privacy-preserving deep learning

Privacy Threats and Protection in Machine Learning

Deep Learning: Differential Privacy Preservation in the Era of Big Data

Privacy-Preserving Deep Learning on Machine Learning as a Service—a Comprehensive Survey

PDLM: Privacy-Preserving Deep Learning Model on Cloud with Multiple Keys

What can we learn from Data Leakage and Unlearning for Law?

Federated Deep Learning with Bayesian Privacy

When Machine Learning Meets Privacy: A Survey and Outlook

Privacy-Preserving Machine Learning Algorithms for Big Data Systems

Research on the Application of Deep Learning-based Differential Privacy Protection Models in Financial Big Data

A privacy preserve big data analysis system for wearable wireless sensor network

Privacy in Large Language Models: Attacks, Defenses and Future Directions

Federated Learning Privacy: Attacks, Defenses, Applications, and Policy Landscape - A Survey

Learning Privacy-Preserving Student Networks via Discriminative-Generative Distillation

Security and Privacy Issues in Deep Learning: A Brief Review

Privacy enhancing and generalizable deep learning with synthetic data for mediastinal neoplasm diagnosis