Abstract:Large Language Models are trained on extensive datasets that often contain sensitive, human-generated information, raising significant concerns about privacy breaches. While certified unlearning approaches offer strong privacy guarantees, they rely on restrictive model assumptions that are not applicable to LLMs. As a result, various unlearning heuristics have been proposed, with the associated privacy risks assessed only empirically. The standard evaluation pipelines typically randomly select data for removal from the training set, apply unlearning techniques, and use membership inference attacks to compare the unlearned models against models retrained without the to-be-unlearned data. However, since every data point is subject to the right to be forgotten, unlearning should be considered in the worst-case scenario from the privacy perspective. Prior work shows that data outliers may exhibit higher memorization effects. Intuitively, they are harder to be unlearn and thus the privacy risk of unlearning them is underestimated in the current evaluation. In this paper, we leverage minority data to identify such a critical flaw in previously widely adopted evaluations. We substantiate this claim through carefully designed experiments, including unlearning canaries related to minority groups, inspired by privacy auditing literature. Using personally identifiable information as a representative minority identifier, we demonstrate that minority groups experience at least 20% more privacy leakage in most cases across six unlearning approaches, three MIAs, three benchmark datasets, and two LLMs of different scales. Given that the right to be forgotten should be upheld for every individual, we advocate for a more rigorous evaluation of LLM unlearning methods. Our minority-aware evaluation framework represents an initial step toward ensuring more equitable assessments of LLM unlearning efficacy.

Inexact Unlearning Needs More Careful Evaluations to Avoid a False Sense of Privacy

When Machine Unlearning Jeopardizes Privacy

FP 2 -MIA: A Membership Inference Attack Free of Posterior Probability in Machine Unlearning.

Silver Linings in the Shadows: Harnessing Membership Inference for Machine Unlearning

Adversarial Machine Unlearning

Underestimated Privacy Risks for Minority Populations in Large Language Model Unlearning

Learn What You Want to Unlearn: Unlearning Inversion Attacks against Machine Unlearning

Towards Reliable Empirical Machine Unlearning Evaluation: A Game-Theoretic View

Gone but Not Forgotten: Improved Benchmarks for Machine Unlearning

Pseudo Unlearning via Sample Swapping with Hash

A Duty to Forget, a Right to Be Assured? Exposing Vulnerabilities in Machine Unlearning Services

Unlearn and Burn: Adversarial Machine Unlearning Requests Destroy Model Accuracy

Game-Theoretic Machine Unlearning: Mitigating Extra Privacy Leakage

Textual Unlearning Gives a False Sense of Unlearning

Verification of Machine Unlearning is Fragile

On the Necessity of Auditable Algorithmic Definitions for Machine Unlearning

UnUnlearning: Unlearning is not sufficient for content regulation in advanced generative AI

Learn to Unlearn: A Survey on Machine Unlearning

Pseudo-Probability Unlearning: Towards Efficient and Privacy-Preserving Machine Unlearning

Discriminative Adversarial Unlearning

A More Practical Approach to Machine Unlearning