研究人员推出了OpenRCA 2.0，这是一个旨在严格评估LLM代理根本原因分析（RCA）能力的新基准。与仅标记最终根本原因的先前数据集不同，OpenRCA 2.0包含分步因果注释，重建了从原因到症状的传播路径。这一名为PAVE的新协议显示，当前前沿的LLM只能在约20.7%的情况下识别出确切的根本原因集，这凸显了它们在验证因果路径方面能力存在的重大缺陷。 AI
影响 该基准突显了LLM代理在复杂推理和工具使用能力方面的关键差距，可能为未来的模型开发提供指导。
排序理由 该集群描述了一篇介绍用于评估LLM能力的基准的新学术论文。
在 Hugging Face Daily Papers 阅读 →
AI 生成摘要 · Google Gemini · 来自 3 个来源。 我们如何撰写摘要 →
arXiv:2606.27154v1 Announce Type: new Abstract: Root cause analysis (RCA) poses a holistic test of LLM agentic capabilities, such as long-context understanding, multi-step reasoning, and tool use. However, existing datasets suffer from a fundamental gap: they label only the root …
Root cause analysis (RCA) poses a holistic test of LLM agentic capabilities, such as long-context understanding, multi-step reasoning, and tool use. However, existing datasets suffer from a fundamental gap: they label only the root cause, not the propagation path connecting it to…
Root cause analysis (RCA) poses a holistic test of LLM agentic capabilities, such as long-context understanding, multi-step reasoning, and tool use. However, existing datasets suffer from a fundamental gap: they label only the root cause, not the propagation path connecting it to…