研究人员开发了一种新理论，解释了深度Transformer的内部工作原理，将其视为执行分布式推理的均场相互作用系统。该理论引入了“函数向量”作为内部状态表示，使Transformer能够通过其层以渐进精细的尺度推断潜在上下文变量。研究表明，Transformer的深度和前馈块能够实现比以往更复杂的上下文学习算法。 AI
影响 为理解和潜在改进深度Transformer模型的上下文学习能力提供了理论框架。
排序理由 该集群包含一篇详细介绍理解AI模型架构新理论框架的学术论文。
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
arXiv:2606.16694v1 Announce Type: cross Abstract: Transformers are widely used as a general-purpose substrate for learning complex correlations between a large collection of coupled variables, but their internal mechanisms have remained mysterious. We introduce a theory of a deep…
Transformers are widely used as a general-purpose substrate for learning complex correlations between a large collection of coupled variables, but their internal mechanisms have remained mysterious. We introduce a theory of a deep transformer as a mean-field interacting system th…