Second-Order Information Matters: Revisiting Machine Unlearning for Large Language Models

Kang Gu,Md Rafi Ur Rashid,Najrin Sultana,Shagufta Mehnaz
2024-03-14
Abstract:With the rapid development of Large Language Models (LLMs), we have witnessed intense competition among the major LLM products like ChatGPT, LLaMa, and Gemini. However, various issues (e.g. privacy leakage and copyright violation) of the training corpus still remain underexplored. For example, the Times sued OpenAI and Microsoft for infringing on its copyrights by using millions of its articles for training. From the perspective of LLM practitioners, handling such unintended privacy violations can be challenging. Previous work addressed the ``unlearning" problem of LLMs using gradient information, while they mostly introduced significant overheads like data preprocessing or lacked robustness. In this paper, contrasting with the methods based on first-order information, we revisit the unlearning problem via the perspective of second-order information (Hessian). Our unlearning algorithms, which are inspired by classic Newton update, are not only data-agnostic/model-agnostic but also proven to be robust in terms of utility preservation or privacy guarantee. Through a comprehensive evaluation with four NLP datasets as well as a case study on real-world datasets, our methods consistently show superiority over the first-order methods.
Machine Learning,Artificial Intelligence,Computation and Language
What problem does this paper attempt to address?
### Problems Addressed by the Paper The paper primarily explores how to effectively handle data deletion requests in large language models (LLMs) and proposes a new method based on second-order information to improve existing first-order methods. Specifically, the paper aims to address the following issues: 1. **Data Privacy and Copyright Infringement**: - With the development of large language models, issues of privacy leaks and copyright infringement in datasets have become increasingly severe. For example, The Times sued OpenAI and Microsoft for using a large number of articles from The Times for training. 2. **Limitations of Existing Methods**: - Existing data "forgetting" methods mostly rely on gradient information, but these methods often require extensive data preprocessing or lack robustness. 3. **Efficient Forgetting Algorithm**: - Proposes a new method based on second-order information (Hessian) to improve the forgetting process, making it not only data-independent and model-independent but also more robust in preserving model utility. By comparing first-order methods (such as gradient ascent), the paper demonstrates the superior performance of the second-order information-based method on multiple natural language processing datasets and further validates its effectiveness on real-world datasets. Additionally, the paper explores the relationship between forgetting algorithms and differential privacy, revealing that differential privacy cannot replace the role of forgetting algorithms.