Abstract:Large Language Models are trained on extensive datasets that often contain sensitive, human-generated information, raising significant concerns about privacy breaches. While certified unlearning approaches offer strong privacy guarantees, they rely on restrictive model assumptions that are not applicable to LLMs. As a result, various unlearning heuristics have been proposed, with the associated privacy risks assessed only empirically. The standard evaluation pipelines typically randomly select data for removal from the training set, apply unlearning techniques, and use membership inference attacks to compare the unlearned models against models retrained without the to-be-unlearned data. However, since every data point is subject to the right to be forgotten, unlearning should be considered in the worst-case scenario from the privacy perspective. Prior work shows that data outliers may exhibit higher memorization effects. Intuitively, they are harder to be unlearn and thus the privacy risk of unlearning them is underestimated in the current evaluation. In this paper, we leverage minority data to identify such a critical flaw in previously widely adopted evaluations. We substantiate this claim through carefully designed experiments, including unlearning canaries related to minority groups, inspired by privacy auditing literature. Using personally identifiable information as a representative minority identifier, we demonstrate that minority groups experience at least 20% more privacy leakage in most cases across six unlearning approaches, three MIAs, three benchmark datasets, and two LLMs of different scales. Given that the right to be forgotten should be upheld for every individual, we advocate for a more rigorous evaluation of LLM unlearning methods. Our minority-aware evaluation framework represents an initial step toward ensuring more equitable assessments of LLM unlearning efficacy.

Silver Linings in the Shadows: Harnessing Membership Inference for Machine Unlearning

When Machine Unlearning Jeopardizes Privacy

Game-Theoretic Machine Unlearning: Mitigating Extra Privacy Leakage

Gone but Not Forgotten: Improved Benchmarks for Machine Unlearning

Learn What You Want to Unlearn: Unlearning Inversion Attacks against Machine Unlearning

Adversarial Machine Unlearning

Learn to Unlearn: A Survey on Machine Unlearning

Pseudo-Probability Unlearning: Towards Efficient and Privacy-Preserving Machine Unlearning

Enhancing Privacy Protection for Online Learning Resource Recommendation with Machine Unlearning

Machine Learning with Membership Privacy using Adversarial Regularization

A Duty to Forget, a Right to Be Assured? Exposing Vulnerabilities in Machine Unlearning Services

FP 2 -MIA: A Membership Inference Attack Free of Posterior Probability in Machine Unlearning.

Inexact Unlearning Needs More Careful Evaluations to Avoid a False Sense of Privacy

Membership Inference via Backdooring

Zero-shot Class Unlearning via Layer-wise Relevance Analysis and Neuronal Path Perturbation

Underestimated Privacy Risks for Minority Populations in Large Language Model Unlearning

Releasing Malevolence from Benevolence: The Menace of Benign Data on Machine Unlearning

Deletion inference, reconstruction, and compliance in machine (un)learning

Machine Unlearning with Minimal Gradient Dependence for High Unlearning Ratios

Fair Machine Unlearning: Data Removal while Mitigating Disparities

Machine unlearning in brain-inspired neural network paradigms