AI peer review policies vary by field; LLMs show biases in quality

A new study published on arXiv investigates the use and regulation of AI in academic peer review. Researchers surveyed AI policies across 111 AI/NLP conferences and medical journals, finding significant differences between the two fields. The study also evaluated AI-generated reviews for ICLR 2026 and Nature Communications, revealing that while current LLMs can produce detailed reviews, they often exhibit biases such as overly positive recommendations and generic criticism. The authors advocate for multi-dimensional evaluation of AI reviews, as aggregate quality scores can be misleading. AI

IMPACT This research highlights the need for careful evaluation of AI-generated peer reviews and suggests potential biases that could affect scientific publishing.

RANK_REASON The cluster contains a research paper detailing findings on AI in academic peer review. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• Alexander Fichtl M.Sc.
• arXiv
• Hugging Face
• ICLR 2026
• Nature Communications

• paper
• policy
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →