本文系统地探究了Transformer模型的上下文学习能力，重点关注高斯混合二分类任务。文章实证分析了输入维度、示例数量和预训练任务等因素如何影响上下文准确率。该研究还探讨了良性过拟合现象，即模型在记忆了嘈杂的上下文标签后仍能良好泛化，并绘制了上下文学习成功或失败的条件图。 AI
影响 提供了上下文分类缩放行为的经验图谱，突出了成功的关键因素。
排序理由 学术论文，探究Transformer模型的上下文学习能力。
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
arXiv:2604.25858v1 Announce Type: new Abstract: Transformers have demonstrated a strong ability for in-context learning (ICL), enabling models to solve previously unseen tasks using only example input output pairs provided at inference time. While prior theoretical work has estab…
Transformers have demonstrated a strong ability for in-context learning (ICL), enabling models to solve previously unseen tasks using only example input output pairs provided at inference time. While prior theoretical work has established conditions under which transformers can p…