研究人员开发了ToolFailBench，这是一个新的诊断基准，旨在识别大型语言模型（LLM）代理中特定的工具使用失败。该基准在1000个跨领域任务中评估代理，将失败分为诸如Tool-Skip（跳过工具）、Result-Ignore（忽略结果）和Output-Fabrication（伪造输出）等类型。此外，还引入了一个名为Vera的独立框架，用于对大型语言模型代理进行自动化安全测试，该框架利用三阶段流程来发现和验证代理行为中的风险，特别是在代理与外部工具交互时。Vera在四个代理框架上的评估揭示了显著的安全弱点，攻击成功率高达93.9%。 AI
影响 突出了大型语言模型代理可靠性和安全性方面的关键改进领域，特别是在工具使用和自主行动方面。
排序理由 该集群包含两篇学术论文，介绍了用于评估大型语言模型代理的新基准和框架。
AI 生成摘要 · Google Gemini · 来自 5 个来源。 我们如何撰写摘要 →
arXiv:2607.07405v1 Announce Type: new Abstract: Tool-using LLM agents can violate the very policies they are deployed to enforce while appearing to complete the task successfully. In policy-permissive environments, a tool may execute any well-formed call even when the correspondi…
Tool-using LLM agents can violate the very policies they are deployed to enforce while appearing to complete the task successfully. In policy-permissive environments, a tool may execute any well-formed call even when the corresponding state transition is forbidden by domain polic…
arXiv:2607.04686v1 Announce Type: cross Abstract: Tool calling is central to modern language model agents, but aggregate benchmark scores often hide where tool use fails. A model that never calls a needed tool and a model that calls the tool but ignores the result can look simila…
Tool calling is central to modern language model agents, but aggregate benchmark scores often hide where tool use fails. A model that never calls a needed tool and a model that calls the tool but ignores the result can look similar under final task accuracy. We introduce ToolFail…
Automated safety testing framework Vera uses a three-stage pipeline to identify and test safety risks in LLM agents through structured risk taxonomies, combinatorial case generation, and adaptive sandbox execution with evidence-based verification.