两篇新的arXiv论文探讨了Transformer中上下文内学习（ICL）的理论基础。一篇论文展示了Transformer如何通过在每一层内隐式执行归一化梯度下降步骤来执行上下文内逻辑回归。另一篇论文研究了非线性回归，展示了注意力机制如何构建特征，使Transformer能够在不更新权重的情况下从示例中学习。 AI
影响 这些论文推进了对Transformer如何从提示中学习的理论理解，可能指导未来的模型开发和优化。
排序理由 两篇arXiv论文对Transformer中的上下文内学习机制进行了理论分析。
AI 生成摘要 · Google Gemini · 来自 4 个来源。 我们如何撰写摘要 →
arXiv:2605.06609v1 Announce Type: new Abstract: Transformers have demonstrated remarkable in-context learning (ICL) capabilities. The strong ICL performance of transformers is commonly believed to arise from their ability to implicitly execute certain algorithms on the context, t…
arXiv:2605.05176v1 Announce Type: new Abstract: Pre-trained transformers are able to learn from examples provided as part of the prompt without any weight updates, a remarkable ability known as in-context learning (ICL). Despite its demonstrated efficacy across various domains, t…
Pre-trained transformers are able to learn from examples provided as part of the prompt without any weight updates, a remarkable ability known as in-context learning (ICL). Despite its demonstrated efficacy across various domains, the theoretical understanding of ICL is still dev…
Transformers have demonstrated remarkable in-context learning (ICL) capabilities. The strong ICL performance of transformers is commonly believed to arise from their ability to implicitly execute certain algorithms on the context, thereby enhancing prediction and generation. In t…