Generative models learn rules across two distinct training timescales

Researchers have identified two distinct timescales in generative model training: the point at which generations become rule-valid ($\tau_{\mathrm{rule}}$) and the point at which models begin reproducing training samples ($\tau_{\mathrm{mem}}$). The interval between these, termed the 'innovation window,' widens with larger datasets and narrows with increased rule complexity. This phenomenon, observed in both diffusion and autoregressive models, explains when and how these models demonstrate genuine innovation. AI

Summary written by gemini-2.5-flash-lite from 1 source. How we write summaries →

IMPACT Provides a theoretical framework for understanding generative model innovation and potential limitations.

RANK_REASON The cluster contains a new academic paper detailing research findings on generative models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• Generative models
• Diffusion models
• Autoregressive models
• arXiv

• paper
• model release