一项新的研究论文强调，由于裁判配置选择的不同，AI安全基准的结果存在显著差异。研究发现，仅改变提示词的措辞，在保持裁判模型不变的情况下，就可能使测得的有害响应率发生高达24.2个百分点的变化。这种敏感性影响了模型安全排名的稳定性，类别级别的差异高达39.6个百分点。研究强调，用于LLM裁判的提示词的具体措辞是影响安全评估的一个关键但被忽视的因素。 AI
影响 揭示了当前AI安全基准可能因提示词敏感性而不可靠，需要更稳健的评估方法。
排序理由 学术论文，详细介绍了AI安全基准方法论的新发现。
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
arXiv:2604.24074v1 Announce Type: new Abstract: Safety benchmarks such as HarmBench rely on LLM judges to classify model responses as harmful or safe, yet the judge configuration, namely the combination of judge model and judge prompt, is typically treated as a fixed implementati…
Safety benchmarks such as HarmBench rely on LLM judges to classify model responses as harmful or safe, yet the judge configuration, namely the combination of judge model and judge prompt, is typically treated as a fixed implementation detail. We show this assumption is problemati…