多篇研究论文正在评估大型语言模型（LLM）在学术同行评审中的能力和局限性。'Review Arcade' 和 'PRAIB' 等研究调查了 LLM 与人类审稿人的一致性，发现 LLM 虽然有帮助，但表现出过度自信和评分变异性较低等偏见。'PRISM' 引入了一个基准来评估 LLM 审稿人在分析深度和缺陷识别等方面的表现，结论是 LLM 在特定领域表现出色，但缺乏人类审稿人的均衡表现。此外，关于 'TADDLE' 的研究侧重于检测 LLM 生成的评审中的不足之处，这些评审通常流畅但可能包含细微错误。一个突出的重大问题是 LLM 容易受到提示注入攻击，这会操纵评审结果。 AI
影响 将 LLM 集成到学术同行评审中需要仔细校准和安全措施，以防范偏见和对抗性攻击，确保可靠性。
排序理由 多篇研究论文引入了新的基准和分析框架，用于评估 LLM 在学术同行评审中的表现。
AI 生成摘要 · Google Gemini · 来自 10 个来源。 我们如何撰写摘要 →
arXiv:2605.28897v1 Announce Type: new Abstract: LLM-generated reviews for scientific papers are gaining considerable traction and are even being officially piloted by major conferences. We have to assume that not only reviewers are using LLM-assistance, but also that authors use …
arXiv:2605.29815v1 Announce Type: new Abstract: The growing number of submitted papers has motivated the exploration of Large Language Models (LLMs) as a means to support and augment the peer review process, particularly in terms of improving its speed and scalability. Yet, it re…
The growing number of submitted papers has motivated the exploration of Large Language Models (LLMs) as a means to support and augment the peer review process, particularly in terms of improving its speed and scalability. Yet, it remains unknown whether LLMs engage with scientifi…
LLM-generated reviews for scientific papers are gaining considerable traction and are even being officially piloted by major conferences. We have to assume that not only reviewers are using LLM-assistance, but also that authors use LLMs to revise their papers before submitting. I…
arXiv:2605.26730v1 Announce Type: new Abstract: The rapid growth in submissions to machine learning venues has strained the scientific peer-review system and intensified interest in LLM-based automated peer reviewers. However, how good these systems are actually, especially compa…
arXiv:2605.26911v1 Announce Type: new Abstract: LLM-generated peer reviews are increasingly common at major venues, yet their deficiencies are hard to detect because they are uniformly fluent and well-structured. Existing work either classifies authorship without judging quality,…
PRISM evaluates automated peer review systems across multiple dimensions using argument mining and retrieval-augmented verification, revealing that while LLMs match human performance in specific areas, no system consistently equals human reviewers across all evaluation criteria.
LLM-generated peer reviews are increasingly common at major venues, yet their deficiencies are hard to detect because they are uniformly fluent and well-structured. Existing work either classifies authorship without judging quality, or scores quality with features designed for hu…
arXiv:2605.25415v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used in academic peer review, yet their reliability, alignment with human judgment, and robustness to adversarial attacks remain poorly understood. We present a systematic benchmark of L…
Large language models (LLMs) are increasingly used in academic peer review, yet their reliability, alignment with human judgment, and robustness to adversarial attacks remain poorly understood. We present a systematic benchmark of LLM-as-a-Reviewer on 898 papers stratified from N…