English(EN) Decoding in Order-Agnostic Language Models: Chain-Rule Deviation and Uniform Spreading

顺序无关语言模型显示出顺序依赖的似然度

作者 PulseAugur 编辑部 · [1 个来源] · 2026-06-02 04:00

研究人员发现，顺序无关语言模型（OALMs）不能完美地分解联合分布，这意味着标记的揭示顺序会影响生成的似然度，最高可达0.49 nats/token。虽然置信度优先解码是顺序无关的，但其标记揭示顺序与从左到右生成非常相似。该研究还提出了一种基于置信度轨迹方差的新诊断工具，表明均匀的置信度扩散最大化了目标可恢复性，并且较低的方差与较高的下游正确性相关。 AI

影响揭示了顺序无关语言模型的根本局限性，可能指导未来解码策略和模型评估的研究。

排序理由学术论文，详细介绍了关于语言模型行为的新发现。[lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.CL 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 0 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 学术论文，详细介绍了关于语言模型行为的新发现。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: paper, model release; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: 128 days old; Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.

完整方法见我们的编辑标准。

报道来源 [1]

arXiv cs.CL TIER_1 English(EN) · Lin Yao · 2026-06-02 04:00

无序语言模型中的解码：链式法则偏差与均匀扩散

arXiv:2606.00997v1 Announce Type: new Abstract: Order-agnostic language models (OALMs), including discrete diffusion language models (dLLMs), are trained to predict masked tokens under arbitrary conditioning sets, allowing sequences to be generated or scored under arbitrary revea…

本文如何被排名

报道来源 [1]

无序语言模型中的解码：链式法则偏差与均匀扩散

相关实体

相关话题