研究人员正在探索神经网络中的“grokking”现象，即模型在开始泛化之前会先记住数据。一项研究提出修改架构拓扑，例如强制执行球形约束或使用均匀注意力，以绕过记忆阶段并加速泛化。另一篇论文利用持久同调来识别一个独特的拓扑信号——同调性的急剧增加——标志着向泛化过渡，为分析表示学习提供了一个新框架。 AI
影响 这些研究通过分析架构拓扑和表示学习，为理解和潜在地加速神经网络泛化提供了新的理论框架。
排序理由 两篇 arXiv 论文利用拓扑和架构修改来研究神经网络中的“grokking”现象。
AI 生成摘要 · Google Gemini · 来自 3 个来源。 我们如何撰写摘要 →
arXiv:2605.06352v1 Announce Type: new Abstract: We study the grokking phenomenon through the lens of topology. Using persistent homology on point clouds derived from the embedding matrices of a range of models trained on modular arithmetic with varying primes, we identify a clear…
arXiv:2603.05228v3 Announce Type: replace Abstract: Mechanistic interpretability typically relies on post-hoc analysis of trained networks. We instead adopt an interventional approach: testing hypotheses a priori by modifying architectural topology to observe training dynamics. W…
We study the grokking phenomenon through the lens of topology. Using persistent homology on point clouds derived from the embedding matrices of a range of models trained on modular arithmetic with varying primes, we identify a clear and consistent topological signature of grokkin…