English(EN) Multiplication Beyond Groups: Stratified Fourier Mechanisms in Transformer Circuits

Transformer电路通过局部代数区域学习模块化乘法

作者 PulseAugur 编辑部 · [2 个来源] · 2026-07-08 06:49

一篇新研究论文探讨了Transformer模型如何学习模块化整数乘法，这是一种复杂且不可逆的操作。该研究提出了一种“幺半群扩展”方法，认为Transformer将输入空间划分为局部代数区域，而不是依赖于单一的全局表示。这使得它们能够在这些区域内应用类似群的结构和傅里叶机制，这可以通过嵌入组织和注意力路由模式得到证明。 AI

影响为理解Transformer执行复杂算法推理的方式提供了见解，可能为未来的模型架构提供信息。

排序理由在arXiv上发表的研究论文，详细介绍了Transformer电路中的新机制。

在 arXiv cs.AI 阅读 →

AI 生成摘要 · Google Gemini · 来自 2 个来源。我们如何撰写摘要 →

报道来源 [2]

arXiv cs.AI TIER_1 English(EN) · Zitong Andrew Chen, Junaid Hasan, Akhil Srinivasan, Hemkesh Bandi, Jarod Alper · 2026-07-09 04:00

Multiplication Beyond Groups: Stratified Fourier Mechanisms in Transformer Circuits

arXiv:2607.07066v1 Announce Type: cross Abstract: Transformers have demonstrated a remarkable ability to learn algorithmic reasoning, yet mechanistic analyses have mostly focused on globally invertible operations such as cyclic addition and group composition. In this work, we inv…
arXiv cs.AI TIER_1 English(EN) · Jarod Alper · 2026-07-08 06:49

Multiplication Beyond Groups: Stratified Fourier Mechanisms in Transformer Circuits

Transformers have demonstrated a remarkable ability to learn algorithmic reasoning, yet mechanistic analyses have mostly focused on globally invertible operations such as cyclic addition and group composition. In this work, we investigate how small transformers learn modular inte…

报道来源 [2]

Multiplication Beyond Groups: Stratified Fourier Mechanisms in Transformer Circuits

Multiplication Beyond Groups: Stratified Fourier Mechanisms in Transformer Circuits

相关实体

相关话题