A recent audit of the IFBench leaderboard on Hugging Face revealed significant issues with data verification and score reliability. As of November 19, 2026, none of the 23 listed entries were officially verified, and a substantial portion originated from unmerged pull requests. Furthermore, the scoring granularity of 300 prompts is insufficient to reliably differentiate between top-ranked models, with the top six models falling within a narrow performance band. AI
IMPACT Highlights the challenges in reliably evaluating and comparing LLM performance due to insufficient data verification and statistical significance.
RANK_REASON Analysis of a benchmark's leaderboard integrity and scoring methodology. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →