New SorryBench™ Benchmark Measures AI Model Apologies

By PulseAugur Editorial · [1 sources] · 2026-07-08 09:44

A new benchmark called SorryBench™ has been introduced to measure how often AI models apologize during productive sessions. The creator noted that existing benchmarks like MMLU, SWE-bench, and ARC-AGI do not capture this specific aspect of model behavior. This benchmark is based on personal observation and is described as being as rigorous as some charts found on current model cards. AI

IMPACT Introduces a novel metric for AI model evaluation, focusing on politeness rather than pure performance.

RANK_REASON The item introduces a new benchmark for evaluating AI models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on Mastodon — fosstodon.org →

other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →

New SorryBench™ Benchmark Measures AI Model Apologies

COVERAGE [1]

Mastodon — fosstodon.org TIER_1 English(EN) · [email protected] · 2026-07-08 09:44

So many benchmarks. MMLU, SWE-bench, ARC-AGI, etc. Yet none of them measure what I actually notice at work. Introducing: SorryBench™. The basis: how often does

So many benchmarks. MMLU, SWE-bench, ARC-AGI, etc. Yet none of them measure what I actually notice at work. Introducing: SorryBench™. The basis: how often does a model apologize per productive session? Method: I worked, and I counted. n=1, vibes-based, not peer-reviewed. About as…

COVERAGE [1]

So many benchmarks. MMLU, SWE-bench, ARC-AGI, etc. Yet none of them measure what I actually notice at work. Introducing: SorryBench™. The basis: how often does

RELATED ENTITIES

RELATED TOPICS