English(EN) So many benchmarks. MMLU, SWE-bench, ARC-AGI, etc. Yet none of them measure what I actually notice at work. Introducing: SorryBench™. The basis: how often does

新的 SorryBench™ 基准测试衡量 AI 模型道歉次数

作者 PulseAugur 编辑部 · [1 个来源] · 2026-07-08 09:44

一个名为 SorryBench™ 的新基准测试被引入，用于衡量 AI 模型在生产性会话中道歉的频率。创建者指出，现有的基准测试，如 MMLU、SWE-bench 和 ARC-AGI，未能捕捉到模型行为的这一特定方面。该基准测试基于个人观察，并被描述为与当前模型卡上的一些图表一样严谨。 AI

影响引入了一种新颖的 AI 模型评估指标，侧重于礼貌而非纯粹的性能。

排序理由该项目引入了一个新的 AI 模型评估基准测试。 [lever_c_demoted from research: ic=1 ai=1.0]

在 Mastodon — fosstodon.org 阅读 →

其他

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

报道来源 [1]

Mastodon — fosstodon.org TIER_1 English(EN) · [email protected] · 2026-07-08 09:44

So many benchmarks. MMLU, SWE-bench, ARC-AGI, etc. Yet none of them measure what I actually notice at work. Introducing: SorryBench™. The basis: how often does

So many benchmarks. MMLU, SWE-bench, ARC-AGI, etc. Yet none of them measure what I actually notice at work. Introducing: SorryBench™. The basis: how often does a model apologize per productive session? Method: I worked, and I counted. n=1, vibes-based, not peer-reviewed. About as…

报道来源 [1]

So many benchmarks. MMLU, SWE-bench, ARC-AGI, etc. Yet none of them measure what I actually notice at work. Introducing: SorryBench™. The basis: how often does

相关实体

相关话题