Abstract:Membership inference (MI) attacks threaten user privacy through determining if a given data example has been used to train a target model. Existing MI defenses protect the membership privacy through preemptive exclusion of members techniques and knowledge distillation. Unfortunately, using either of these two defenses alone, the defense effect can still offers an unsatisfactory trade-off between membership privacy and utility.Given that the defense method that directly combines these two defenses is still very limited (e.g., the test accuracy of the target model is decreased by about 40% (in our experiments)), in this work, we propose a Dual Defense (DD) method that includes the preemptive exclusion of high-risk member samples module and the knowledge distillation module, which thwarts the access of the resulting models to the private training data twice to mitigate MI attacks. Our defense method can be divided into two steps: the preemptive exclusion of high-risk member samples (Step 1) and the knowledge distillation to obtain the protected student model (Step 2). We propose three types of exclusions: existing MI attacks-based exclusions, sample distances of members and nonmembers-based exclusions, and mutual information value-based exclusions, to preemptively exclude the high-risk member samples. During the knowledge distillation phase, we add ground-truth labeled data to the reference dataset to decrease the protected student model’s dependency on soft labels, aiming to maintain or improve its test accuracy. Extensive evaluation shows that DD significantly outperforms state-of-the-art defenses and offers a better privacy-utility trade-off. For example, DD achieves ∼100% test accuracy improvement over the DMP defense [1] for ResNet50 trained on CIFAR100. DD simultaneously achieves the reductions in the attack effectiveness (e.g., the TPR@0.01%FPR of Enhanced MI attacks [2] decreased by 2.10% on the ImageNet dataset, the membership advantage (MA) of Risk score-based attacks [3] decreased by 56.30%) and improvements of the target models’ test accuracies (e.g., by 42.80% on CIFAR100).

Understanding and defending against White-box membership inference attack in deep learning

Defending Privacy Against More Knowledgeable Membership Inference Attackers

Unveiling the Unseen: Exploring Whitebox Membership Inference through the Lens of Explainability

Adversarial for Good – Defending Training Data Privacy with Adversarial Attack Wisdom

A GAN-Based Defense Framework Against Model Inversion Attacks.

Private Knowledge Transfer via Model Distillation with Generative Adversarial Networks

Defenses to Membership Inference Attacks: A Survey

Membership reconstruction attack in deep neural networks

Dual Defense: Combining Preemptive Exclusion of Members and Knowledge Distillation to Mitigate Membership Inference Attacks

Privacy Analysis of Deep Learning in the Wild: Membership Inference Attacks against Transfer Learning

Defending Against Membership Inference Attacks: RM Learning is All You Need

White-box Membership Inference Attacks against Diffusion Models

Membership Privacy for Machine Learning Models Through Knowledge Transfer

Diffence: Fencing Membership Privacy With Diffusion Models

MIST: Defending Against Membership Inference Attacks Through Membership-Invariant Subspace Training

A Method to Facilitate Membership Inference Attacks in Deep Learning Models

CALoR: Towards Comprehensive Model Inversion Defense

Privacy Leakage on DNNs: A Survey of Model Inversion Attacks and Defenses

Practical Blind Membership Inference Attack via Differential Comparisons

Deep Neural Network Quantization Framework for Effective Defense against Membership Inference Attacks

Membership Inference Attacks and Defenses in Classification Models