Abstract:The arms race between attacks and defenses for machine learning models has come to a forefront in recent years, in both the security community and the privacy community. However, one big limitation of previous research is that the security domain and the privacy domain have typically been considered separately. It is thus unclear whether the defense methods in one domain will have any unexpected impact on the other domain. In this paper, we take a step towards resolving this limitation by combining the two domains. In particular, we measure the success of membership inference attacks against six state-of-the-art defense methods that mitigate the risk of adversarial examples (i.e., evasion attacks). Membership inference attacks determine whether or not an individual data record has been part of a model's training set. The accuracy of such attacks reflects the information leakage of training algorithms about individual members of the training set. Adversarial defense methods against adversarial examples influence the model's decision boundaries such that model predictions remain unchanged for a small area around each input. However, this objective is optimized on training data. Thus, individual data records in the training set have a significant influence on robust models. This makes the models more vulnerable to inference attacks. To perform the membership inference attacks, we leverage the existing inference methods that exploit model predictions. We also propose two new inference methods that exploit structural properties of robust models on adversarially perturbed data. Our experimental evaluation demonstrates that compared with the natural training (undefended) approach, adversarial defense methods can indeed increase the target model's risk against membership inference attacks. When using adversarial defenses to train the robust models, the membership inference advantage increases by up to 4.5 times compared to the naturally undefended models. Beyond revealing the privacy risks of adversarial defenses, we further investigate the factors, such as model capacity, that influence the membership information leakage.

Explaining the Model, Protecting Your Data: Revealing and Mitigating the Data Privacy Risks of Post-Hoc Model Explanations via Membership Inference

Private Data Inference Attacks against Cloud: Model, Technologies, and Research Directions

Inferring Sensitive Attributes from Model Explanations

Accurate, Explainable, and Private Models: Providing Recourse While Minimizing Training Data Leakage

Model Explanations with Differential Privacy

Attribute inference privacy protection for pre-trained models

A Survey of Privacy-Preserving Model Explanations: Privacy Risks, Attacks, and Countermeasures

Privacy Risks of Securing Machine Learning Models against Adversarial Examples

Label-Only Membership Inference Attack Based on Model Explanation

Privacy Backdoors: Enhancing Membership Inference through Poisoning Pre-trained Models

Unveiling the Unseen: Exploring Whitebox Membership Inference through the Lens of Explainability

Translating theory into practice: assessing the privacy implications of concept-based explanations for biomedical AI

The privacy-explainability trade-off: unraveling the impacts of differential privacy and federated learning on attribution methods

PreCurious: How Innocent Pre-Trained Language Models Turn into Privacy Traps

TMI! Finetuned Models Leak Private Information from their Pretraining Data

SoK: Reducing the Vulnerability of Fine-tuned Language Models to Membership Inference Attacks

Systematic Evaluation of Privacy Risks of Machine Learning Models

Explaining Vulnerabilities to Adversarial Machine Learning through Visual Analytics

A simple defense against adversarial attacks on heatmap explanations

Privacy-Preserving Case-Based Explanations: Enabling Visual Interpretability by Protecting Privacy

Overconfidence is a Dangerous Thing: Mitigating Membership Inference Attacks by Enforcing Less Confident Prediction