研究人员开发了ParaEval，一个旨在改进大型语言模型评估的新框架。当前的单项选择题问答基准对答案的具体措辞过于敏感，导致对模型真实知识的评估不准确。ParaEval通过使用多个释义的答案选项来查询模型来解决这个问题，从而提供一种更强大的能力衡量标准，而不是仅仅熟悉特定的短语。 AI
影响 提供了一种更可靠的评估LLM知识的方法，可能导致更准确的模型开发和比较。
排序理由 该集群包含一篇提出LLM新评估方法的学术论文。
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
arXiv:2606.10657v1 Announce Type: new Abstract: Multiple-choice (MCQA) benchmarks are the standard for evaluating pretrained large language models, but their reliance on log-likelihood scoring makes them unreliable. Specifically, standard scores are highly sensitive to the exact …
Multiple-choice (MCQA) benchmarks are the standard for evaluating pretrained large language models, but their reliance on log-likelihood scoring makes them unreliable. Specifically, standard scores are highly sensitive to the exact phrasing (surface form) of the answers, conflati…