AI shouldn't automate peer review without rigorous evaluation, paper argues

A new paper argues against the immediate automation of academic peer review using current large language models. The research highlights two major issues: AI reviewers exhibit an excessive agreement, limiting diverse perspectives, and their scores can be easily manipulated through stylistic paper rewrites rather than genuine scientific merit. The authors propose that a dedicated science of peer review automation is necessary, rather than deploying general-purpose LLMs without thorough evaluation. AI

影响 Current LLMs are not suitable for automating peer review due to lack of diversity and susceptibility to manipulation, necessitating specialized research.

排序理由 Academic paper evaluating the use of LLMs in peer review. [lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.AI 阅读 →

• arXiv
• ICLR
• LLM

• 论文
• 安全
• 其他

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →