PulseAugur
中
实时 08:18:19
English(EN) The Review Lottery: Benchmarking an Observational Estimator of Peer-Review Noise (ICLR 2017-2025)

AI会议同行评审质量分析：分歧率高，未发现下降 · 跟踪2个来源

作者 PulseAugur 编辑部 · [2 个来源] ·

两篇新的arXiv论文分析了AI会议同行评审的质量和一致性。第一篇论文《评审彩票》使用2017-2025年ICLR数据的观察性估计器，发现存在显著的分歧率，表明23-30%的论文在更换审稿人后，其录用/拒绝决定可能会被翻转。第二篇论文《同行评审真的在衰落吗？》分析了ICLR、NeurIPS和ACL，开发了一个新的框架来量化评审质量。该研究反驳了衰落的论调，发现中位数评审质量随时间没有持续下降，并为相关担忧提出了替代性解释。 AI

影响 这些研究为AI领域科学评估的可靠性提供了关键见解，可能影响未来的评审流程以及AI研究的可信度。

排序理由 两篇在arXiv上发表的学术论文，分析了AI会议的同行评审质量。

在 arXiv cs.AI 阅读 →

AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →

AI会议同行评审质量分析：分歧率高，未发现下降 · 跟踪2个来源

本文如何被排名

Signal score
34 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Research
两篇在arXiv上发表的学术论文，分析了AI会议的同行评审质量。
Source corroboration
2 independent sources
Multiple independent publishers reporting the same story raises confidence that it's real and newsworthy.
Topics
paper, other
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
Breaking (< 6h)
Fresh story with cross-source coverage still developing. Ranking may shift as more sources report.

完整方法见我们的编辑标准。

报道来源 [2]

  1. arXiv cs.AI TIER_1 English(EN) · Feilian Huang ·

    审稿抽签：对同行评审噪音的观察性估计器的基准测试 (ICLR 2017-2025)

    arXiv:2610.06591v2 Announce Type: replace Abstract: How much of a conference accept/reject decision would change if the same paper were reviewed by a different set of reviewers? Running a second independent program committee is the gold standard for answering this, but it is proh…

  2. arXiv cs.CL TIER_1 English(EN) · Ilia Kuznetsov, Rohan Nayak, Alla Rozovskaya, Iryna Gurevych ·

    同行评审真的在衰落吗？分析各场馆和时间的评审质量

    arXiv:2601.15172v2 Announce Type: replace Abstract: Peer review is at the heart of modern science. As submission numbers rise and research communities grow, the decline in review quality is a popular narrative and a common concern. Yet, is it true? Review quality is difficult to …

相关实体

相关话题