Abstract:The trustworthy machine learning (ML) community is increasingly recognizing the crucial need for models capable of selectively 'unlearning' data points after training. This leads to the problem of machine unlearning (MU), aiming to eliminate the influence of chosen data points on model performance, while still maintaining the model's utility post-unlearning. Despite various MU methods for data influence erasure, evaluations have largely focused on random data forgetting, ignoring the vital inquiry into which subset should be chosen to truly gauge the authenticity of unlearning performance. To tackle this issue, we introduce a new evaluative angle for MU from an adversarial viewpoint. We propose identifying the data subset that presents the most significant challenge for influence erasure, i.e., pinpointing the worst-case forget set. Utilizing a bi-level optimization principle, we amplify unlearning challenges at the upper optimization level to emulate worst-case scenarios, while simultaneously engaging in standard training and unlearning at the lower level, achieving a balance between data influence erasure and model utility. Our proposal offers a worst-case evaluation of MU's resilience and effectiveness. Through extensive experiments across different datasets (including CIFAR-10, 100, CelebA, Tiny ImageNet, and ImageNet) and models (including both image classifiers and generative models), we expose critical pros and cons in existing (approximate) unlearning strategies. Our results illuminate the complex challenges of MU in practice, guiding the future development of more accurate and robust unlearning algorithms. The code is available at <a class="link-external link-https" href="https://github.com/OPTML-Group/Unlearn-WorstCase" rel="external noopener nofollow">this https URL</a>.

Accurate Forgetting for All-in-One Image Restoration Model

Targeted Therapy in Data Removal: Object Unlearning Based on Scene Graphs

Attack and Reset for Unlearning: Exploiting Adversarial Noise toward Machine Unlearning through Parameter Re-initialization

Learning with Recoverable Forgetting

Active forgetting via influence estimation for neural networks

Boundary Unlearning: Rapid Forgetting of Deep Networks Via Shifting the Decision Boundary

Deep Unlearning: Fast and Efficient Gradient-free Approach to Class Forgetting

Boundary Unlearning

Only My Model On My Data: A Privacy Preserving Approach Protecting one Model and Deceiving Unauthorized Black-Box Models

Scissorhands: Scrub Data Influence via Connection Sensitivity in Networks

Mixed-Privacy Forgetting in Deep Networks

Unlearn and Burn: Adversarial Machine Unlearning Requests Destroy Model Accuracy

DeepClean: Machine Unlearning on the Cheap by Resetting Privacy Sensitive Weights using the Fisher Diagonal

Machine Unlearning for Image Retrieval: A Generative Scrubbing Approach

Learning to Unlearn: Instance-wise Unlearning for Pre-trained Classifiers

Learn to Forget: Machine Unlearning Via Neuron Masking

All-In-One Image Restoration for Unknown Corruption

An Information Theoretic Approach to Machine Unlearning

Pseudo-Probability Unlearning: Towards Efficient and Privacy-Preserving Machine Unlearning

Restore Anything with Masks: Leveraging Mask Image Modeling for Blind All-in-One Image Restoration

Challenging Forgets: Unveiling the Worst-Case Forget Sets in Machine Unlearning