两篇新研究论文探讨了位置编码（PE）在Transformer模型中的必要性。其中一篇论文证明，滑动窗口Transformer在没有PE的情况下也能实现图灵完备性，这表明窗口机制本身提供了足够的位置信息。另一篇论文研究了PE在Vision Transformers（ViTs）中的作用，发现虽然ViTs在没有PE的情况下也能发展出空间结构，但PE可以锚定这种结构并显著提高其在内容干扰性分布变化下的鲁棒性。 AI
影响 挑战了位置编码的必要性，可能简化未来的Transformer架构并提高效率。
排序理由 两篇在arXiv上发表的学术论文，讨论了Transformer架构的理论方面。
在 Hugging Face Daily Papers 阅读 →
AI 生成摘要 · Google Gemini · 来自 3 个来源。 我们如何撰写摘要 →
arXiv:2606.01532v1 Announce Type: new Abstract: Positional encoding (PE) is widely viewed as necessary for transformers to process ordered sequences: without them, the next-token map appears permutation-invariant in its context tokens. This intuition underlies all prior universal…
arXiv:2606.00124v1 Announce Type: cross Abstract: Positional embeddings (PEs) in Vision Transformers (ViTs) are known to impact performance and robustness, but their role in shaping internal spatial representations is not well understood. In this work, we study how different form…
Positional encoding (PE) is widely viewed as necessary for transformers to process ordered sequences: without them, the next-token map appears permutation-invariant in its context tokens. This intuition underlies all prior universality results, which rely on positional informatio…