研究人员推出了 DiagFlowBench，这是一个新的数据集，旨在评估语言模型在诊断对话中处理非流程或超出范围输入的性能。该数据集包含 1,676 个多轮对话，源自 50 个工业诊断流程图，揭示了一个漏洞：模型可能会选择看似合理但错误的步骤，而不是选择不执行任何操作。对十个商业和开源模型的评估显示，它们识别和恰当回应超出范围查询的能力存在显著差异，这表明在实际维护操作中对模型进行接地存在挑战。 AI
影响 突出了接地 LLM 的一个关键漏洞，可能影响其在工业维护和咨询角色中的可靠性。
排序理由 该集群包含一篇学术论文，详细介绍了用于评估语言模型的新基准数据集。
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
完整方法见我们的编辑标准。
arXiv:2606.17904v1 Announce Type: new Abstract: Language models increasingly serve as advisory systems in maintenance operations. To prevent hallucination, recent systems ground these models in procedural documentation to constrain them to approved steps. In practice, however, op…
Language models increasingly serve as advisory systems in maintenance operations. To prevent hallucination, recent systems ground these models in procedural documentation to constrain them to approved steps. In practice, however, operator queries frequently stray from this path, …