Abstract:Large Language Models (LLMs) are advancing at a remarkable pace, with myriad applications under development. Unlike most earlier machine learning models, they are no longer built for one specific application but are designed to excel in a wide range of tasks. A major part of this success is due to their huge training datasets and the unprecedented number of model parameters, which allow them to memorize large amounts of information contained in the training data. This memorization goes beyond mere language, and encompasses information only present in a few documents. This is often desirable since it is necessary for performing tasks such as question answering, and therefore an important part of learning, but also brings a whole array of issues, from privacy and security to copyright and beyond. LLMs can memorize short secrets in the training data, but can also memorize concepts like facts or writing styles that can be expressed in text in many different ways. We propose a taxonomy for memorization in LLMs that covers verbatim text, facts, ideas and algorithms, writing styles, distributional properties, and alignment goals. We describe the implications of each type of memorization - both positive and negative - for model performance, privacy, security and confidentiality, copyright, and auditing, and ways to detect and prevent memorization. We further highlight the challenges that arise from the predominant way of defining memorization with respect to model behavior instead of model weights, due to LLM-specific phenomena such as reasoning capabilities or differences between decoding algorithms. Throughout the paper, we describe potential risks and opportunities arising from memorization in LLMs that we hope will motivate new research directions.

Detecting Unintended Memorization in Language-Model-Fused ASR

Unintended Memorization in Large ASR Models, and How to Mitigate It

Detecting Memorization in Large Language Models

Mitigating Memorization In Language Models

Uncovering Latent Memories: Assessing Data Leakage and Memorization Patterns in Frontier AI Models

MemHunter: Automated and Verifiable Memorization Detection at Dataset-scale in LLMs

Undesirable Memorization in Large Language Models: A Survey

Mitigating Unintended Memorization in Language Models via Alternating Teaching

Be like a Goldfish, Don't Memorize! Mitigating Memorization in Generative LLMs

Predicting and analyzing memorization within fine-tuned Large Language Models

Preventing Verbatim Memorization in Language Models Gives a False Sense of Privacy

Unlocking Memorization in Large Language Models with Dynamic Soft Prompting

Understanding Memorisation in LLMs: Dynamics, Influencing Factors, and Implications

Rethinking LLM Memorization through the Lens of Adversarial Compression

Emergent and Predictable Memorization in Large Language Models

SoK: Memorization in General-Purpose Large Language Models

Counterfactual Memorization in Neural Language Models

Recite, Reconstruct, Recollect: Memorization in LMs as a Multifaceted Phenomenon

Quantifying and Analyzing Entity-level Memorization in Large Language Models

Mitigating Approximate Memorization in Language Models via Dissimilarity Learned Policy

Demystifying Verbatim Memorization in Large Language Models