Generative models learn rules across two distinct training timescales

Researchers have identified two distinct timescales in generative model training: the point at which generations become rule-valid ($\tau_{\mathrm{rule}}$) and the point at which models begin reproducing training samples ($\tau_{\mathrm{mem}}$). The interval between these, termed the 'innovation window,' widens with larger datasets and narrows with increased rule complexity. This phenomenon, observed in both diffusion and autoregressive models, explains when and how these models demonstrate genuine innovation. AI

IMPACT Provides a theoretical framework for understanding generative model innovation and potential limitations.

RANK_REASON The cluster contains a new academic paper detailing research findings on generative models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• arXiv
• Diffusion models
• Generative models

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →