研究人员推出了QUACK，这是一个开源环境和评估框架，旨在审计大型语言模型（LLM）代理在多模态社交推理任务中使用的语言基础。QUACK根据游戏结果、行为轨迹和话语级一致性来评估代理，特别标记空间幻觉和无根据指控等问题。对三个前沿VLM的评估显示出重大的基础失败，代理幻觉的空间声明超过15%，并且超过一半的指控没有证据支持。 AI
影响 该框架通过识别和纠正语言基础问题，可以提高LLM代理在复杂推理任务中的可靠性，从而使其更加健壮。
排序理由 该集群描述了一篇介绍LLM代理评估框架的新研究论文。
在 Hugging Face Daily Papers 阅读 →
AI 生成摘要 · Google Gemini · 来自 3 个来源。 我们如何撰写摘要 →
完整方法见我们的编辑标准。
arXiv:2605.27068v1 Announce Type: cross Abstract: Social deduction games have become a popular testbed for probing reasoning, deception, coordination, and belief modeling in Large Language Model (LLM) agents. However, most environments are scored only by game outcomes such as win…
Social deduction games have become a popular testbed for probing reasoning, deception, coordination, and belief modeling in Large Language Model (LLM) agents. However, most environments are scored only by game outcomes such as win rates and largely remain to text-only interaction…
A multimodal social reasoning environment and evaluation framework called QUACK is introduced to audit the grounding of agent language through three-level assessment of game outcomes, behavioral trajectories, and utterance-level consistency.