New benchmark Tampered ML tests LLMs on spotting flaws in AI research

A new benchmark called Tampered ML has been developed to evaluate the ability of large language models (LLMs) to identify flaws in machine learning research proposals. The benchmark consists of 1,858 pairs of experiment proposals, where one is a minimally edited version of a published proposal containing a methodological flaw. Human validation confirmed the benchmark's validity, and current frontier models, including Opus 5, show performance above chance but are far from reliable. These models also exhibit biases, favoring flawed proposals that appear more specific, transparent, or conventional, and are sensitive to the order in which proposals are presented. AI

IMPACT Highlights the current limitations of LLMs in reliably assessing the quality of AI research, indicating a need for further development in AI alignment and critical evaluation capabilities.

RANK_REASON The cluster describes a new academic paper introducing a benchmark for evaluating LLMs. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• alphaXiv
• arXiv
• CatalyzeX
• DagsHub
• Gotit.pub
• Hugging Face
• LLM
• machine learning research
• ScienceCast
• Tampered ML

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →