English(EN) Investigation into In-Context Learning Capabilities of Transformers

研究人员探究Transformer上下文学习的缩放和过拟合

作者 PulseAugur 编辑部 · [2 个来源] · 2026-04-28 16:57

本文系统地探究了Transformer模型的上下文学习能力，重点关注高斯混合二分类任务。文章实证分析了输入维度、示例数量和预训练任务等因素如何影响上下文准确率。该研究还探讨了良性过拟合现象，即模型在记忆了嘈杂的上下文标签后仍能良好泛化，并绘制了上下文学习成功或失败的条件图。 AI

影响提供了上下文分类缩放行为的经验图谱，突出了成功的关键因素。

排序理由学术论文，探究Transformer模型的上下文学习能力。

AI 生成摘要 · Google Gemini · 来自 2 个来源。我们如何撰写摘要 →

报道来源 [2]

arXiv cs.LG TIER_1 English(EN) · Rushil Chandrupatla, Leo Bangayan, Sebastian Leng, Arya Mazumdar · 2026-04-29 04:00

Transformer模型上下文学习能力调查

arXiv:2604.25858v1 Announce Type: new Abstract: Transformers have demonstrated a strong ability for in-context learning (ICL), enabling models to solve previously unseen tasks using only example input output pairs provided at inference time. While prior theoretical work has estab…
arXiv cs.AI TIER_1 English(EN) · Arya Mazumdar · 2026-04-28 16:57

Transformer模型上下文学习能力探究

Transformers have demonstrated a strong ability for in-context learning (ICL), enabling models to solve previously unseen tasks using only example input output pairs provided at inference time. While prior theoretical work has established conditions under which transformers can p…