BadScientist: Can a Research Agent Write Convincing but Unsound Papers that Fool LLM Reviewers?
Researchers have developed a framework called BadScientist to test the vulnerability of AI-driven peer review systems to fabricated research papers. The system uses presentation-manipulation strategies without conducting real experiments, and it was found that these fabricated papers achieved high acceptance rates when reviewed by LLM-based systems. Despite reviewers flagging integrity issues, they often still assigned acceptance scores, indicating a significant concern-acceptance conflict. Mitigation strategies showed only marginal improvements, highlighting fundamental limitations in current AI review processes and the need for robust safeguards. AI
IMPACT Highlights critical vulnerabilities in AI-driven peer review systems, underscoring the need for human oversight and advanced defense mechanisms in scientific publishing.