一篇新的研究论文对“大型语言模型在知识经济任务上始终表现出专家级人类水平”的叙事提出了质疑。该研究强调，当前的基准测试常常未能考虑到训练数据重叠问题，也未能充分衡量错误的大小或响应的可靠性。通过引入一项新颖的基于编码的数据分析任务，研究发现人类专家在平均表现上优于前沿的大型语言模型，表现出较低的性能变异性和较少的重大错误。 AI
影响 强调需要超越平均性能指标的更稳健的大型语言模型评估方法。
排序理由 该集群包含一篇讨论大型语言模型性能局限性的学术论文。
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
arXiv:2606.11166v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly described as performing at the level of human experts on knowledge economy tasks. These claims are primarily based on how LLMs perform on benchmarking tasks that measure average perfor…
Large Language Models (LLMs) are increasingly described as performing at the level of human experts on knowledge economy tasks. These claims are primarily based on how LLMs perform on benchmarking tasks that measure average performance across standardized datasets. Primary limita…