研究人员将模型崩溃（大型语言模型在训练自身输出来进行训练时会退化）这一现象重新解读为一种文化演化过程。通过应用迭代学习理论，他们使用LLaMA-2-7B和Mistral-7B模型在多种语言上推导并测试了五个预测。一个关键发现是，在未经筛选的自训练过程中，组合性最初会增加然后减少，这种模式即使在正则化数据下也持续存在，并且只有通过任务基础的筛选才能缓解。 AI
影响 为理解和缓解模型崩溃提供了一个新的理论视角，可能改进自训练管道设计。
排序理由 该集群包含一篇学术论文，详细介绍了针对已知AI现象的新理论框架和实验验证。
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
arXiv:2605.23054v1 Announce Type: cross Abstract: Model collapse, the progressive degradation of LLMs trained on their own outputs, has been characterized statistically but lacks a linguistic explanation for which structures degrade, in what order, and why. We show that iterated …
Model collapse, the progressive degradation of LLMs trained on their own outputs, has been characterized statistically but lacks a linguistic explanation for which structures degrade, in what order, and why. We show that iterated learning theory from cultural evolution fills this…