Abstract:Machine unlearning aims to remove information derived from forgotten data while preserving that of the remaining dataset in a well-trained model. With the increasing emphasis on data privacy, several approaches to machine unlearning have emerged. However, these methods typically rely on complete supervision throughout the unlearning process. Unfortunately, obtaining such supervision, whether for the forgetting or remaining data, can be impractical due to the substantial cost associated with annotating real-world datasets. This challenge prompts us to propose a supervision-free unlearning approach that operates without the need for labels during the unlearning process. Specifically, we introduce a variational approach to approximate the distribution of representations for the remaining data. Leveraging this approximation, we adapt the original model to eliminate information from the forgotten data at the representation level. To further address the issue of lacking supervision information, which hinders alignment with ground truth, we introduce a contrastive loss to facilitate the matching of representations between the remaining data and those of the original model, thus preserving predictive performance. Experimental results across various unlearning tasks demonstrate the effectiveness of our proposed method, Label-Agnostic Forgetting (LAF) without using any labels, which achieves comparable performance to state-of-the-art methods that rely on full supervision information. Furthermore, our approach excels in semi-supervised scenarios, leveraging limited supervision information to outperform fully supervised baselines. This work not only showcases the viability of supervision-free unlearning in deep models but also opens up a new possibility for future research in unlearning at the representation level.

Label-Agnostic Forgetting: A Supervision-Free Unlearning in Deep Models

A few-shot Label Unlearning in Vertical Federated Learning

LLM Unlearning via Loss Adjustment with Only Forget Data

Decoupling the Class Label and the Target Concept in Machine Unlearning

Unlearning from Weakly Supervised Learning

Federated Unlearning via Active Forgetting

Machine Unlearning of Features and Labels

An Information Theoretic Approach to Machine Unlearning

A Closer Look at Machine Unlearning for Large Language Models

A Duty to Forget, a Right to Be Assured? Exposing Vulnerabilities in Machine Unlearning Services

Deep Unlearning: Fast and Efficient Gradient-free Approach to Class Forgetting

Loss-Free Machine Unlearning

Attribute-to-Delete: Machine Unlearning via Datamodel Matching

Can Bad Teaching Induce Forgetting? Unlearning in Deep Networks Using an Incompetent Teacher

Unlearn What You Want to Forget: Efficient Unlearning for LLMs

Boundary Unlearning: Rapid Forgetting of Deep Networks Via Shifting the Decision Boundary

Boundary Unlearning

CaMU: Disentangling Causal Effects in Deep Model Unlearning

Certified Machine Unlearning via Noisy Stochastic Gradient Descent

Machine Unlearning on Pre-trained Models by Residual Feature Alignment Using LoRA

Challenging Forgets: Unveiling the Worst-Case Forget Sets in Machine Unlearning