On the limits and opportunities of AI reviewers: Reviewing the reviews of Nature-family papers with 45 expert scientists
A new study evaluated AI reviewers against human experts in assessing scientific papers, finding that AI models like GPT-5.2, Gemini 3.0 Pro, and Claude Opus 4.5 can outperform top human reviewers on certain metrics. While AI reviewers identified unique issues and were rated highly for correctness and evidence, they also exhibited weaknesses such as limited subfield knowledge and excessive overlap in their critiques. The research concludes that current AI reviewers are best utilized as complements to human expertise rather than replacements. AI
IMPACT AI reviewers show potential to augment human expertise in scientific publishing, identifying unique issues but requiring oversight for consistency and depth.