Abstract:Among existing privacy attacks on the gradient of neural networks, data reconstruction attack, which reverse engineers the training batch from the gradient, poses a severe threat on the private training data. Despite its empirical success on large architectures and small training batches, unstable reconstruction accuracy is also observed when a smaller architecture or a larger batch is under attack. Due to the weak interpretability of existing learning-based attacks, there is little known on why, when and how data reconstruction attack is feasible. In our work, we perform the first analytic study on the security boundary of data reconstruction from gradient via a microcosmic view on neural networks with rectified linear units (ReLUs), the most popular activation function in practice. For the first time, we characterize the insecure/secure boundary of data reconstruction attack in terms of the neuron exclusivity state of a training batch, indexed by the number of Exclusively Activated Neurons (ExANs, i.e., a ReLU activated by only one sample in a batch). Intuitively, we show a training batch with more ExANs are more vulnerable to data reconstruction attack and vice versa. On the one hand, we construct a novel deterministic attack algorithm which substantially outperforms previous attacks for reconstructing training batches lying in the insecure boundary of a neural network. Meanwhile, for training batches lying in the secure boundary, we prove the impossibility of unique reconstruction, based on which an exclusivity reduction strategy is devised to enlarge the secure boundary for mitigation purposes.

Bounding data reconstruction attacks with the hypothesis testing interpretation of differential privacy

Bounding Reconstruction Attack Success of Adversaries Without Data Priors

Bounding Training Data Reconstruction in DP-SGD

Analyzing Privacy Leakage in Machine Learning via Multiple Hypothesis Testing: A Lesson From Fano

Defending against Reconstruction Attacks with Rényi Differential Privacy

Data Reconstruction Attacks and Defenses: A Systematic Evaluation

Visual Privacy Auditing with Diffusion Models

Efficient and Private Marginal Reconstruction with Local Non-Negativity

Renyi Differential Privacy in the Shuffle Model: Enhanced Amplification Bounds

Bayes' capacity as a measure for reconstruction attacks in federated learning

Personalized Differential Privacy for Ridge Regression

Statistical Inference in the Differential Privacy Model

Beyond the Calibration Point: Mechanism Comparison in Differential Privacy

The Limits of Differential Privacy in Online Learning

Reconciling privacy and accuracy in AI for medical imaging

From Robustness to Privacy and Back

Mechanisms for Robust Local Differential Privacy

Randomization Resilient To Sensitive Reconstruction

Wasserstein Differential Privacy

Does Differential Privacy Prevent Backdoor Attacks in Practice?

Exploring the Security Boundary of Data Reconstruction via Neuron Exclusivity Analysis