New theory explores model collapse from LLM-generated text replay

A new theoretical framework, "Language Generation with Replay: A Learning-Theoretic View of Model Collapse," examines the problem of model collapse in large language models (LLMs). The research introduces a "replay adversary" to analyze how generated text re-entering training corpora can degrade performance. The study finds that while replay is benign for uniform generation, it can limit non-uniform generation and generation in the limit, highlighting potential failures of practical heuristics like data cleaning and watermarking. AI

IMPACT Provides a theoretical understanding of potential performance degradation in LLMs due to training data contamination.

RANK_REASON Academic paper on a theoretical aspect of LLM training. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv stat.ML →

• arXiv
• Giorgio Racca
• Hugging Face
• Language Generation with Replay: A Learning-Theoretic View of Model Collapse
• stat.ML

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →