研究人员推出了JuICE，一个旨在评估大型语言模型识别自身回应中文化错误能力的新基准。该数据集包含来自美国、韩国、印度尼西亚和孟加拉国的1,050个查询-回应对的7,470个文化和语言错误标注。测试显示，即使是表现最佳的LLM裁判，在检测错误跨度方面的F1分数也仅为0.52，表明与人类评估者相比，它们在理解细微的文化背景方面存在显著差距。 AI
影响 强调需要更复杂的评估方法，以确保LLM在不同全球用户中具有文化适应性。
排序理由 该集群描述了一篇介绍用于评估LLM能力的基准数据集的新学术论文。
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
arXiv:2605.26955v1 Announce Type: cross Abstract: As large language models (LLMs) are increasingly deployed to users around the world, they are integrated into everyday tasks across diverse cultural contexts, from drafting personal communications to brainstorming creative ideas. …
As large language models (LLMs) are increasingly deployed to users around the world, they are integrated into everyday tasks across diverse cultural contexts, from drafting personal communications to brainstorming creative ideas. These tasks are inherently cultural: they require …