Abstract:Among existing privacy attacks on the gradient of neural networks, data reconstruction attack, which reverse engineers the training batch from the gradient, poses a severe threat on the private training data. Despite its empirical success on large architectures and small training batches, unstable reconstruction accuracy is also observed when a smaller architecture or a larger batch is under attack. Due to the weak interpretability of existing learning-based attacks, there is little known on why, when and how data reconstruction attack is feasible. In our work, we perform the first analytic study on the security boundary of data reconstruction from gradient via a microcosmic view on neural networks with rectified linear units (ReLUs), the most popular activation function in practice. For the first time, we characterize the insecure/secure boundary of data reconstruction attack in terms of the neuron exclusivity state of a training batch, indexed by the number of Exclusively Activated Neurons (ExANs, i.e., a ReLU activated by only one sample in a batch). Intuitively, we show a training batch with more ExANs are more vulnerable to data reconstruction attack and vice versa. On the one hand, we construct a novel deterministic attack algorithm which substantially outperforms previous attacks for reconstructing training batches lying in the insecure boundary of a neural network. Meanwhile, for training batches lying in the secure boundary, we prove the impossibility of unique reconstruction, based on which an exclusivity reduction strategy is devised to enlarge the secure boundary for mitigation purposes.

Reconstructing Training Data from Model Gradient, Provably

The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networks

Quantification of the Leakage in Federated Learning

Theory-Oriented Deep Leakage from Gradients Via Linear Equation Solver.

Understanding Training-Data Leakage from Gradients in Neural Networks for Image Classification

Seeing the Forest through the Trees: Data Leakage from Partial Transformer Gradients

Recover User's Private Training Image Data by Gradient in Federated Learning

Exploring the Security Boundary of Data Reconstruction via Neuron Exclusivity Analysis

Gradient leakage attacks in federated learning

R-CONV: An Analytical Approach for Efficient Data Reconstruction via Convolutional Gradients

Stealing Secrecy from Outside: A Novel Gradient Inversion Attack in Federated Learning

Gradient-Leaks: Enabling Black-Box Membership Inference Attacks Against Machine Learning Models

Leak and Learn: An Attacker's Cookbook to Train Using Leaked Data from Federated Learning

Understanding Deep Gradient Leakage via Inversion Influence Functions

A Theoretical Insight into Attack and Defense of Gradient Leakage in Transformer

Optimal Defenses Against Gradient Reconstruction Attacks

Training Data Reconstruction: Privacy due to Uncertainty?

Gradient Leakage Defense with Key-Lock Module for Federated Learning

Is Diffusion Model Safe? Severe Data Leakage via Gradient-Guided Diffusion Model

Gradient Inversion Attacks: Impact Factors Analyses and Privacy Enhancement

Dropout is NOT All You Need to Prevent Gradient Leakage