Generative models learn rules across two distinct training timescales

Researchers have identified two distinct timescales in generative model training: the point at which generations become rule-valid ($\tau_{\mathrm{rule}}$) and the point at which models begin reproducing training samples ($\tau_{\mathrm{mem}}$). The interval between these, termed the 'innovation window,' widens with larger datasets and narrows with increased rule complexity. This phenomenon, observed in both diffusion and autoregressive models, explains when and how these models demonstrate genuine innovation. AI

影响 Provides a theoretical framework for understanding generative model innovation and potential limitations.

排序理由 The cluster contains a new academic paper detailing research findings on generative models. [lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.AI 阅读 →

• arXiv
• Diffusion models
• Generative models

• 论文
• 模型发布

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →