两篇新研究论文探讨了大语言模型中上下文学习的机制。一篇论文研究了是否可以使用Transformer激活来优化上下文样本选择，发现MLP输出与性能不相关，并提出了稀疏自编码器等未来研究方向。另一篇论文提出，自注意力层和MLP层的堆叠使Transformer能够根据上下文隐式更新MLP权重，可能在无需额外训练的情况下解释上下文学习能力。 AI
影响 这些论文为大语言模型如何从提示中学习提供了理论见解，可能指导未来的模型开发和微调策略。
排序理由 两篇在arXiv上发表的学术论文，探讨了大语言模型中上下文学习的技术基础。
AI 生成摘要 · Google Gemini · 来自 3 个来源。 我们如何撰写摘要 →
arXiv:2606.05134v1 Announce Type: new Abstract: Deep active learning has previously been explored for LLM in-context sample selection, but not with methods that utilise recent advances in understanding of transformer activations. In this paper, we test the hypothesis that model a…
Deep active learning has previously been explored for LLM in-context sample selection, but not with methods that utilise recent advances in understanding of transformer activations. In this paper, we test the hypothesis that model activations could provide a fine-grained signal t…
arXiv:2507.16003v4 Announce Type: replace Abstract: One of the most striking features of Large Language Models (LLMs) is their ability to learn in-context. Namely at inference time an LLM is able to learn new patterns without any additional weight update when these patterns are p…