Abstract:The proliferation of generative models, combined with pretraining on web-scale data, raises a timely question: what happens when these models are trained on their own generated outputs? Recent investigations into model-data feedback loops proposed that such loops would lead to a phenomenon termed model collapse, under which performance progressively degrades with each model-data feedback iteration until fitted models become useless. However, those studies largely assumed that new data replace old data over time, where an arguably more realistic assumption is that data accumulate over time. In this paper, we ask: what effect does accumulating data have on model collapse? We empirically study this question by pretraining sequences of language models on text corpora. We confirm that replacing the original real data by each generation's synthetic data does indeed tend towards model collapse, then demonstrate that accumulating the successive generations of synthetic data alongside the original real data avoids model collapse; these results hold across a range of model sizes, architectures, and hyperparameters. We obtain similar results for deep generative models on other types of real data: diffusion models for molecule conformation generation and variational autoencoders for image generation. To understand why accumulating data can avoid model collapse, we use an analytically tractable framework introduced by prior work in which a sequence of linear models are fit to the previous models' outputs. Previous work used this framework to show that if data are replaced, the test error increases with the number of model-fitting iterations; we extend this argument to prove that if data instead accumulate, the test error has a finite upper bound independent of the number of iterations, meaning model collapse no longer occurs.

AI models fed AI-generated data quickly spew nonsense

AI produces gibberish when trained on too much AI-generated data

AI models collapse when trained on recursively generated data

Nepotistically Trained Generative-AI Models Collapse

Bigger AI chatbots more inclined to spew nonsense — and people don't always realize

How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapse

Larger and more instructable language models become less reliable

Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data

Generative AI Degrades Online Communities

A Precis of Language Models are not Models of Language

AI and the Problem of Knowledge Collapse

AI & robotics briefing: The year machines will run out of data

Strong Model Collapse

AI & robotics briefing: Why superintelligent AI won't sneak up on us

Testing AI on language comprehension tasks reveals insensitivity to underlying meaning

Use of large language models might affect our cognitive skills

A Note on Shumailov et al. (2024): `AI Models Collapse When Trained on Recursively Generated Data'

A Tale of Tails: Model Collapse as a Change of Scaling Laws

Collapse of Self-trained Language Models

Large Language Models are Not Models of Natural Language: They are Corpus Models