PulseAugur
中
实时 07:50:20

新定理解释了大型语言模型如何随着时间推移忘记提示

研究人员开发了一个理论框架,称为“条件无毛定理”,以解释大型语言模型如何随着时间推移忘记其提示。研究发现,提示对Transformer模型的影响会随着幂律衰减,经过训练的模型比未经训练的模型遗忘得更快。这种现象归因于注意力机制稀释了提示的影响以及生成文本的增长,理论界限表明,在生成文本的条件下,提示的影响会渐近消失。还提供了一个用于衡量Hugging Face模型中遗忘率的工具包。 AI

影响 为理解大型语言模型中的提示衰减提供了理论基础,可能为未来的模型设计和提示策略提供信息。

排序理由 详细介绍大型语言模型行为理论发现的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.AI 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →

新定理解释了大型语言模型如何随着时间推移忘记提示

本文如何被排名

Signal score
27 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Tool
详细介绍大型语言模型行为理论发现的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration
Single-source cluster
Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics
paper, model release
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
Breaking (< 6h)
Fresh story with cross-source coverage still developing. Ranking may shift as more sources report.

完整方法见我们的编辑标准。

报道来源 [1]

  1. arXiv cs.AI TIER_1 English(EN) · Shuyang Xiang ·

    Transformer 的条件无毛定理:语言模型如何忘记其提示

    arXiv:2610.24133v2 Announce Type: replace Abstract: A black hole retains only a few conserved charges of the matter it formed from. We are inspired to make the analogy and we ask how much an autoregressive transformer retains of its prompt after generating $t$ tokens. For two pro…