LLM peer reviewers favor confident rhetoric over scientific substance

A new study published on arXiv explores the potential for linguistic bias in large language models (LLMs) used for academic peer review. Researchers systematically altered the linguistic style of research papers, focusing on native vs. non-native English and high- vs. low-confidence rhetoric, while preserving the core scientific content. The experiments revealed that LLM reviewers tend to favor confident language, regardless of the paper's actual scientific merit, highlighting a significant vulnerability in AI-assisted peer review processes. AI

IMPACT Highlights a critical vulnerability in LLM-based peer review, potentially impacting the integrity of scientific publishing.

RANK_REASON Academic paper detailing a new experimental framework for studying LLM bias. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• arXiv
• large language models
• LLM reviewers
• Sai Suresh Macharla Vasu

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →