Abstract:The open source of large amounts of image data promotes the development of deep learning techniques. Along with this comes the privacy risk of these open-source image datasets being exploited by unauthorized third parties to train deep learning models for commercial or illegal purposes. To avoid the abuse of public data, a poisoning-based technique, the unlearnable example, is proposed to significantly degrade the generalization performance of models by adding a kind of imperceptible noise to the data. To further enhance its robustness against adversarial training, existing works leverage iterative adversarial training on both the defensive noise and the surrogate model. However, it still remains unknown whether the robustness of unlearnable examples primarily comes from the effect of enhancement in the surrogate model or the defensive noise. Observing that simply removing the adversarial noise on the training process of the defensive noise can improve the performance of robust unlearnable examples, we identify that solely the surrogate model's robustness contributes to the performance. Furthermore, we found a negative correlation exists between the robustness of defensive noise and the protection performance, indicating defensive noise's instability issue. Motivated by this, to further boost the robust unlearnable example, we introduce stable error-minimizing noise (SEM), which trains the defensive noise against random perturbation instead of the time-consuming adversarial perturbation to improve the stability of defensive noise. Through extensive experiments, we demonstrate that SEM achieves a new state-of-the-art performance on CIFAR-10, CIFAR-100, and ImageNet Subset in terms of both effectiveness and efficiency. The code is available at <a class="link-external link-https" href="https://github.com/liuyixin-louis/Stable-Unlearnable-Example" rel="external noopener nofollow">this https URL</a>.

MSR: Making Self-supervised learning Robust to Aggressive Augmentations

Learning Where to Learn in Cross-View Self-Supervised Learning

Weak Augmentation Guided Relational Self-Supervised Learning

Can We Break Free from Strong Data Augmentations in Self-Supervised Learning?

Characterizing and Improving the Robustness of Self-Supervised Learning through Background Augmentations

ReSSL: Relational Self-Supervised Learning with Weak Augmentation.

Mind Your Augmentation: the Key to Decoupling Dense Self-Supervised Learning

SASSL: Enhancing Self-Supervised Learning via Neural Style Transfer

Self-supervised Learning is More Robust to Dataset Imbalance

The Common Stability Mechanism behind most Self-Supervised Learning Approaches

De-coupling and De-positioning Dense Self-supervised Learning

Contrastive Learning With Stronger Augmentations

Image-Feature Weak-to-Strong Consistency: An Enhanced Paradigm for Semi-Supervised Learning

Understanding self-supervised Learning Dynamics without Contrastive Pairs

A Simple Baseline for Semi-supervised Semantic Segmentation with Strong Data Augmentation

Augmentations vs Algorithms: What Works in Self-Supervised Learning

On Improving the Algorithm-, Model-, and Data- Efficiency of Self-Supervised Learning

Stable Unlearnable Example: Enhancing the Robustness of Unlearnable Examples via Stable Error-Minimizing Noise

MAST: Masked Augmentation Subspace Training for Generalizable Self-Supervised Priors

From Obstacles to Resources: Semi-supervised Learning Faces Synthetic Data Contamination

Boosting Semi-Supervised 2D Human Pose Estimation by Revisiting Data Augmentation and Consistency Training