AI agent benchmarks flawed: simulated users break rules in 24% of successes

A new benchmark called UserProxyBench, developed by Ashish Jain and Armaan Sandhu, reveals a significant flaw in current agent evaluation methods. The benchmark found that in nearly a quarter of successful runs, the simulated user, often another language model, violated its own instructions. These violations, particularly premature disclosure of information, made the agent's task easier without reflecting its true capability, leading to inflated success scores. The study highlights that the choice of user proxy can drastically alter benchmark results, with User Fidelity Scores ranging from 0.599 to 0.864 across different models. AI

IMPACT Highlights critical flaws in AI agent evaluation, suggesting current benchmarks may overestimate agent capabilities.

RANK_REASON The item describes a new benchmark and its findings regarding evaluation methodologies for AI agents. [lever_c_demoted from research: ic=1 ai=1.0]

Read on dev.to — LLM tag →

• Armaan Sandhu
• Ashish Jain
• Claude Opus 4-8
• Gemini 3.5 Flash
• Gemma 26B
• GPT 5.4 Mini
• GPT-5.5
• GPT 5.6 "Sol"
• John Smith
• sonnet-4-6
• tau-bench
• UserProxyBench

• product
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →