Abstract:Machine learning models leak information about the datasets on which they are trained. An adversary can build an algorithm to trace the individual members of a model's training dataset. As a fundamental inference attack, he aims to distinguish between data points that were part of the model's training set and any other data points from the same distribution. This is known as the tracing (and also membership inference) attack. In this paper, we focus on such attacks against black-box models, where the adversary can only observe the output of the model, but not its parameters. This is the current setting of machine learning as a service in the Internet. We introduce a privacy mechanism to train machine learning models that provably achieve membership privacy: the model's predictions on its training data are indistinguishable from its predictions on other data points from the same distribution. We design a strategic mechanism where the privacy mechanism anticipates the membership inference attacks. The objective is to train a model such that not only does it have the minimum prediction error (high utility), but also it is the most robust model against its corresponding strongest inference attack (high privacy). We formalize this as a min-max game optimization problem, and design an adversarial training algorithm that minimizes the classification loss of the model as well as the maximum gain of the membership inference attack against it. This strategy, which guarantees membership privacy (as prediction indistinguishability), acts also as a strong regularizer and significantly generalizes the model. We evaluate our privacy mechanism on deep neural networks using different benchmark datasets. We show that our min-max strategy can mitigate the risk of membership inference attacks (close to the random guess) with a negligible cost in terms of the classification error.

Adversarial Learning of Privacy-Preserving and Task-Oriented Representations

Adversarial for Good – Defending Training Data Privacy with Adversarial Attack Wisdom

Better accuracy with quantified privacy: representations learned via reconstructive adversarial network.

Privacy Adversarial Network: Representation Learning for Mobile Data Privacy

Adversarial Representation Learning for Robust Privacy Preservation in Audio

Fool Attackers by Imperceptible Noise: A Privacy-Preserving Adversarial Representation Mechanism for Collaborative Learning

Towards Privacy-Preserving Visual Recognition via Adversarial Training: A Pilot Study

Privacy-preserving Adversarial Facial Features

Adversarial Representation Sharing: A Quantitative and Secure Collaborative Learning Framework

Towards Task-Free Privacy-Preserving Data Collection

Learning Robust and Privacy-Preserving Representations via Information Theory

Learning to Anonymize Faces for Privacy Preserving Action Detection

Privacy-Preserving Feature Extraction Via Adversarial Training

Adversarial Perturbation for Privacy Preservation

Machine Learning with Membership Privacy using Adversarial Regularization

Visual Privacy Protection Based on Type-I Adversarial Attack

Transferable Adversarial Facial Images for Privacy Protection

Attribute inference privacy protection for pre-trained models

Using Adversarial Noises to Protect Privacy in Deep Learning Era

Discriminative Adversarial Privacy: Balancing Accuracy and Membership Privacy in Neural Networks

Privacy-Aware Human Mobility Prediction via Adversarial Networks