Abstract:Recently, the enactment of privacy regulations has promoted the rise of the machine unlearning paradigm. Existing studies of machine unlearning mainly focus on sample-wise unlearning, such that a learnt model will not expose user's privacy at the sample level. Yet we argue that such ability of selective removal should also be presented at the attribute level, especially for the attributes irrelevant to the main task, e.g., whether a person recognized in a face recognition system wears glasses or the age range of that person. Through a comprehensive literature review, it is found that existing studies on attribute-related problems like fairness and de-biasing learning cannot address the above concerns properly. To bridge this gap, we propose a paradigm of selectively removing input attributes from feature representations which we name `attribute unlearning'. In this paradigm, certain attributes will be accurately captured and detached from the learned feature representations at the stage of training, according to their mutual information. The particular attributes will be progressively eliminated along with the training procedure towards convergence, while the rest of attributes related to the main task are preserved for achieving competitive model performance. Considering the computational complexity during the training process, we not only give a theoretically approximate training method, but also propose an acceleration scheme to speed up the training process. We validate our method by spanning several datasets and models and demonstrate that our design can preserve model fidelity and reach prevailing unlearning efficacy with high efficiency. The proposed unlearning paradigm builds a foundation for future machine unlearning system and will become an essential component of the latest privacy-related legislation.

Learning to Unlearn: Instance-wise Unlearning for Pre-trained Classifiers

Deep Unlearning: Fast and Efficient Gradient-free Approach to Class Forgetting

Targeted Therapy in Data Removal: Object Unlearning Based on Scene Graphs

Partially Blinded Unlearning: Class Unlearning for Deep Networks a Bayesian Perspective

Challenging Forgets: Unveiling the Worst-Case Forget Sets in Machine Unlearning

Unlearn and Burn: Adversarial Machine Unlearning Requests Destroy Model Accuracy

Machine Unlearning of Features and Labels

Cross-Lingual Unlearning of Selective Knowledge in Multilingual Language Models

Markov Chain Monte Carlo-Based Machine Unlearning: Unlearning What Needs to be Forgotten

Unlearn What You Want to Forget: Efficient Unlearning for LLMs

Towards Robust and Cost-Efficient Knowledge Unlearning for Large Language Models

Don't Forget Too Much: Towards Machine Unlearning on Feature Level

Boundary Unlearning

Boundary Unlearning: Rapid Forgetting of Deep Networks Via Shifting the Decision Boundary

Learning to Unlearn for Robust Machine Unlearning

Efficient Attribute Unlearning: Towards Selective Removal of Input Attributes from Feature Representations

Silver Linings in the Shadows: Harnessing Membership Inference for Machine Unlearning

Unlearning from Weakly Supervised Learning

Class Machine Unlearning for Complex Data via Concepts Inference and Data Poisoning

Generative Adversarial Networks Unlearning

Learn to Unlearn for Deep Neural Networks: Minimizing Unlearning Interference with Gradient Projection