English(EN) Evaluation protocols for historical script classification: the cost of manuscript-level leakage and the role of soft labels

研究揭示手稿级泄露夸大了AI手稿分类的准确性

作者 PulseAugur 编辑部 · [1 个来源] · 2026-11-11 05:00

一篇新论文探讨了历史手稿分类的评估协议，强调了手稿级泄露对准确性指标的重大影响。研究人员发现，在训练手稿的页面上进行测试会极大地夸大感知准确性，从页面级拆分转向手稿级拆分时，准确性从97%下降到54%。该研究还调查了软标签和置信度校准的作用，表明虽然软标签可能不会提高准确性，但它们可以使预测分布更接近专家评估，并减少与专家评估的分歧。 AI

影响强调了在专业领域评估AI模型时需要考虑的关键因素，特别是数据泄露和置信度校准。

排序理由学术论文，详细介绍了特定AI任务的评估协议和研究结果。[lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.CV 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 34 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 学术论文，详细介绍了特定AI任务的评估协议和研究结果。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: paper, other; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: Breaking (< 6h); Fresh story with cross-source coverage still developing. Ranking may shift as more sources report.

完整方法见我们的编辑标准。

报道来源 [1]

arXiv cs.CV TIER_1 English(EN) · Daniil I. Naryzhnyi (ITMO University), Roman A. Popkov (ITMO University), Daniil A. Bakalin (ITMO University) · 2026-11-11 05:00

历史手稿分类的评估协议：手稿级泄露的成本与软标签的作用

arXiv:2611.12969v1 Announce Type: new Abstract: A paleographer applies a script type classifier to unseen manuscripts. We measure how much testing on other pages of training manuscripts inflates accuracy. On VML-HP-ext (715 pages, 171 Hebrew manuscripts, 14 script classes) we com…

本文如何被排名

报道来源 [1]

历史手稿分类的评估协议：手稿级泄露的成本与软标签的作用

相关实体

相关话题