Abstract:While LLMs have demonstrated impressive performance across various domains and tasks, their security issues have become increasingly severe. Machine unlearning (MU) has emerged as a promising solution to address these issues by removing the influence of undesired data on the target model without compromising its utility in other aspects. MU typically assumes full access to the original training data to preserve utility, which is difficult to achieve in LLM unlearning. Existing LLM unlearning methods often assume access to data most affected by undesired data unlearning. However, this assumption underestimates the entanglement among various LLM capabilities and ignores data access limitations due to various issues. Moreover, these LLM unlearning methods do not sufficiently consider that unlearning requests in real-world scenarios are continuously emerging. To overcome these challenges and achieve practical LLM unlearning, we propose the O3 framework. The O3 framework includes an Out-Of-Distribution (OOD) detector to measure the similarity between input and unlearning data, and an Orthogonal low-rank adapter (LoRA) for continuously unlearning requested data. The OOD detector is trained with a novel contrastive entropy loss and utilizes a local-global layer-aggregated scoring mechanism. The orthogonal LoRA achieves parameter disentanglement among continual unlearning requests. During inference, our O3 framework can smartly decide whether and to what extent to load the unlearning LoRA based on the OOD detector's predictions. Notably, O3's effectiveness does not rely on any retained data. We conducted extensive experiments on O3 and state-of-the-art LLM unlearning methods across three tasks and seven datasets. The results indicate that O3 consistently achieves the best trade-off between unlearning effectiveness and utility preservation, especially when facing continuous unlearning requests.

Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning

Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning

SOUL: Unlocking the Power of Second-Order Optimization for LLM Unlearning

Alternate Preference Optimization for Unlearning Factual Knowledge in Large Language Models

Unlearning as multi-task optimization: A normalized gradient difference approach with an adaptive learning rate

Reversing the Forget-Retain Objectives: An Efficient LLM Unlearning Framework from Logit Difference

Challenging Forgets: Unveiling the Worst-Case Forget Sets in Machine Unlearning

As Simple as Fine-tuning: LLM Alignment via Bidirectional Negative Feedback Loss

Learning to Unlearn for Robust Machine Unlearning

UNDIAL: Self-Distillation with Adjusted Logits for Robust Unlearning in Large Language Models

MEOW: MEMOry Supervised LLM Unlearning Via Inverted Facts

Practical Unlearning for Large Language Models

Learn while Unlearn: An Iterative Unlearning Framework for Generative Language Models

WAGLE: Strategic Weight Attribution for Effective and Modular Unlearning in Large Language Models

ULMR: Unlearning Large Language Models Via Negative Response and Model Parameter Average

A Closer Look at Machine Unlearning for Large Language Models

Self-supervised Preference Optimization: Enhance Your Language Model with Preference Degree Awareness

Selective and collaborative influence function for efficient recommendation unlearning

MUNBa: Machine Unlearning via Nash Bargaining