A new paper introduces the Wiggle Framework to stress-test large language model judges across various tasks and axes of stability. The framework evaluates how LLM verdicts change under re-prompting, single challenges, and sustained pressure. Findings indicate that all tested frontier models exhibit instability, with verdicts flipping frequently, and adversarial persuasion often corrupts judgments against ground truth. AI
IMPACT Highlights the unreliability of LLM judges, suggesting caution is needed when using their outputs for critical tasks.
RANK_REASON The cluster describes a new academic paper detailing a framework for evaluating LLM judges. [lever_c_demoted from research: ic=1 ai=1.0]
Read on X — Omar Sanseviero (HF research) →
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
