English(EN) Unmatched Does Not Mean False: Incomplete Reference Sets Can Reverse Calibration Rankings in Open-Ended Theory-of-Mind Tracking

AI评估缺陷：不完整的参考集逆转模型排名

作者 PulseAugur 编辑部 · [1 个来源] · 2026-08-27 04:00

一篇新发表在arXiv上的论文，题为“不匹配不等于错误：不完整的参考集会逆转开放式心智理论追踪的校准排名”（Unmatched Does Not Mean False: Incomplete Reference Sets Can Reverse Calibration Rankings in Open-Theory-of-Mind Tracking），揭示了开放式心智理论（ToM）模型评估中的一个关键缺陷。研究表明，当前依赖有限参考集的评估流程可能会错误地将模型有效的输出标记为错误。这会导致校准排名逆转，即在这些有缺陷的标签下表现较差的模型，在由人类正确性评估时实际上表现更好。 AI

影响突出了当前AI评估方法中的一个关键缺陷，可能影响心智理论模型基准测试结果的可靠性。

排序理由该集群包含一篇详细介绍AI模型评估新发现的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.CL 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 30 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 该集群包含一篇详细介绍AI模型评估新发现的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: paper, safety; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: Breaking (< 6h); Fresh story with cross-source coverage still developing. Ranking may shift as more sources report.

完整方法见我们的编辑标准。

报道来源 [1]

arXiv cs.CL TIER_1 English(EN) · Zhexi Feng, Wuxi Chen, Bingrui Zhang · 2026-08-27 04:00

无与伦比不等于错误：不完整的参考集会逆转开放式心智理论追踪中的校准排名

arXiv:2608.25654v1 Announce Type: new Abstract: Open-ended Theory-of-Mind (ToM) trackers emit valid beliefs absent from finite references. A finite-reference-plus-matcher pipeline marks unmatched outputs false, creating proxy labels that can reverse proper-score model selection o…

本文如何被排名

报道来源 [1]

无与伦比不等于错误：不完整的参考集会逆转开放式心智理论追踪中的校准排名

相关实体

相关话题