一篇题为“稳健性的幻觉”的新研究论文强调了当前大型语言模型评估中的一个关键缺陷。虽然模型在聚合上可能看起来准确，但研究表明，无关的上下文信息会导致单个示例的预测发生重大变化。这种因模型和数据集而异的不稳定性表明，当前的聚合准确性指标可能掩盖了潜在的可靠性问题，因此需要进行逐个示例的评估方法。 AI
影响 强调了对大型语言模型需要更稳健的评估指标，这可能会影响模型性能的评估和部署方式。
排序理由 在arXiv上发表的研究论文，详细介绍了大型语言模型评估方法中的一个缺陷。
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
arXiv:2607.12963v1 Announce Type: new Abstract: As large language models (LLMs) grow more capable, they are increasingly deployed in context-rich settings where task inputs are often accompanied by long, partially irrelevant context. In a controlled setting, we find that state-of…
As large language models (LLMs) grow more capable, they are increasingly deployed in context-rich settings where task inputs are often accompanied by long, partially irrelevant context. In a controlled setting, we find that state-of-the-art models often appear robust to task-irre…