Abstract:Federated learning (FL) enables multiple local clients to collaboratively train a global model, which can reduce privacy leakage by sharing model parameters instead of private datasets. However, recent works have revealed that gradient-based data reconstruction attacks, e.g., deep leakage from gradients (DLG), improved DLG (iDLG), and inverting gradients (IG), may still reveal private information by exploiting model parameters from a local client. Current privacy-preserving FL strategies, e.g., differential privacy or gradient compression, can handle such attacks to some extent, but seriously sacrifice their model accuracy. In this work, we propose a novel privacy-preserving FL method, named privacy-encoded FL (PEFL), to combat such data reconstruction attacks without degrading the model performance. The key concept of PEFL is that each large weight matrix of the neural network model is decomposed into multiple cascading sub-matrices, which thus establishes a novel privacy-encoded mechanism by introducing an entangling nonlinear mapping between model gradients and raw data. As such, multiple sub-matrices are directly trained in parallel at the local clients, but only partial sub-matrices are reported to a global server, which suffices to reconstruct the global model whilst increasing the complexity of the coupling between the model parameters and raw data. We provide a detailed analysis of the accuracy, security, and complexity of our method. As shown, it breaks the limit of classical defensive methods, by significantly reducing the risk of data reconstruction attacks yet not degrading the model performance. Compared to classical defenses, the proposed PEFL decreases the peak-signal-to-noise ratio (PSNR) between the reconstructed data and the raw data by ~ 20dB, without sacrificing the test accuracy. As a new paradigm for privacy-preserving FL, our proposed method has great potential in privacy-demanding learning applications.

Privacy-preserving Collaborative Learning with Scalable Image Transformation and Autoencoder.

Privacy-Preserving Collaborative Deep Learning with Unreliable Participants.

GELU-Net: A Globally Encrypted, Locally Unencrypted Deep Neural Network for Privacy-Preserved Learning.

Privacy-preserving Collaborative Learning with Automatic Transformation Search

Private Knowledge Transfer via Model Distillation with Generative Adversarial Networks

PPCL: Privacy-preserving collaborative learning for mitigating indirect information leakage

Automatic Transformation Search Against Deep Leakage from Gradients

Privacy-Preserving Deep Learning Using Deformable Operators for Secure Task Learning

Privacy-Encoded Federated Learning Against Gradient-Based Data Reconstruction Attacks.

Masked image: Visually protected image dataset privacy-preserving scheme for convolutional neural networks

SPEED: Secure, PrivatE, and Efficient Deep learning

Privacy-Preserving Autoencoder for Collaborative Object Detection

A Lightweight and Accuracy-Lossless Privacy-Preserving Method in Federated Learning

Efficient and Verifiable Privacy-Preserving Federal Learning Framework

Privacy-Preserving Distributed Deep Learning Via Homomorphic Re-Encryption

A Lightweight Privacy-Preserving Scheme Using Label-based Pixel Block Mixing for Image Classification in Deep Learning

A Framework to Preserve User Privacy for Machine Learning as a Service

A Privacy-Preserving Scheme For Convolutional Neural Network-Based Applications In Mobile Cloud

Towards Efficient and Privacy-Preserving Federated Deep Learning