研究人员推出了Sci-Rho，一个旨在测试视觉语言模型（VLM）在STEM问题上鲁棒性的新多语言基准。该基准包含五个学科和七种语言的4200多个问题模板，生成了超过42000个独特实例。对17个最先进VLM的评估显示，平均准确率和最差情况准确率之间存在显著差距，与大型专有模型相比，小型模型在跨语言性能下降方面表现更明显。 AI
影响 强调了对VLM更鲁棒的评估方法的需求，尤其是在不同语言和视觉背景下。
排序理由 该集群包含一篇详细介绍用于评估AI模型的新基准的学术论文。
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
arXiv:2606.08034v1 Announce Type: cross Abstract: Symbolic benchmarks have emerged as a key approach to assess model robustness under minor modifications to STEM-related questions. However, existing symbolic benchmarks mostly remain limited to mathematical reasoning, lack visual …
Symbolic benchmarks have emerged as a key approach to assess model robustness under minor modifications to STEM-related questions. However, existing symbolic benchmarks mostly remain limited to mathematical reasoning, lack visual grounding, and are predominantly in English. In th…