Abstract:The arms race between attacks and defenses for machine learning models has come to a forefront in recent years, in both the security community and the privacy community. However, one big limitation of previous research is that the security domain and the privacy domain have typically been considered separately. It is thus unclear whether the defense methods in one domain will have any unexpected impact on the other domain. In this paper, we take a step towards resolving this limitation by combining the two domains. In particular, we measure the success of membership inference attacks against six state-of-the-art defense methods that mitigate the risk of adversarial examples (i.e., evasion attacks). Membership inference attacks determine whether or not an individual data record has been part of a model's training set. The accuracy of such attacks reflects the information leakage of training algorithms about individual members of the training set. Adversarial defense methods against adversarial examples influence the model's decision boundaries such that model predictions remain unchanged for a small area around each input. However, this objective is optimized on training data. Thus, individual data records in the training set have a significant influence on robust models. This makes the models more vulnerable to inference attacks. To perform the membership inference attacks, we leverage the existing inference methods that exploit model predictions. We also propose two new inference methods that exploit structural properties of robust models on adversarially perturbed data. Our experimental evaluation demonstrates that compared with the natural training (undefended) approach, adversarial defense methods can indeed increase the target model's risk against membership inference attacks. When using adversarial defenses to train the robust models, the membership inference advantage increases by up to 4.5 times compared to the naturally undefended models. Beyond revealing the privacy risks of adversarial defenses, we further investigate the factors, such as model capacity, that influence the membership information leakage.

Lightweight Privacy Protection via Adversarial Sample

Adversarial for Good – Defending Training Data Privacy with Adversarial Attack Wisdom

You Can't Fool All the Models: Detect Adversarial Samples Via Pruning Models.

SHAPER: A General Architecture for Privacy-Preserving Primitives in Secure Machine Learning.

Adversarial Sample Detection Via Channel Pruning

Privacy Risks of Securing Machine Learning Models against Adversarial Examples

SAME: Sample Reconstruction against Model Extraction Attacks

Adversarial Sample-Based Approach for Tighter Privacy Auditing in Final Model-Only Scenarios

DeMem: Privacy-Enhanced Robust Adversarial Learning via De-Memorization

Protection Against Reconstruction and Its Applications in Private Federated Learning

Model Protection: Real-Time Privacy-Preserving Inference Service for Model Privacy at the Edge

Data Lineage Inference: Uncovering Privacy Vulnerabilities of Dataset Pruning

Membership Inference Attacks and Defenses in Neural Network Pruning

PriPrune: Quantifying and Preserving Privacy in Pruned Federated Learning

Privacy for Rescue: A New Testimony Why Privacy is Vulnerable In Deep Models

Adversarial Learning of Privacy-Preserving and Task-Oriented Representations

No Privacy Left Outside: on the (In-)Security of TEE-Shielded DNN Partition for On-Device ML

Second Rethinking of Network Pruning in the Adversarial Setting

Protecting Privacy in Shared Photos Via Adversarial Examples Based Stealth

PATROL: Privacy-Oriented Pruning for Collaborative Inference Against Model Inversion Attacks

Distillation as a Defense to Adversarial Perturbations Against Deep Neural Networks