研究人员推出了KellyBench，这是一个旨在评估语言模型在动态环境中长时序顺序决策能力的新基准。该基准模拟了体育博彩市场，特别是英格兰足球超级联赛，挑战智能体利用历史数据和公开赔率来最大化资金增长。初步评估显示，即使是先进的模型也举步维艰，表现最好的模型平均亏损8%，许多模型则遭遇破产，这表明与人类专家策略相比存在显著差距。 AI
影响 凸显了当前前沿模型在复杂、动态环境中的局限性，表明需要改进适应性策略。
排序理由 该集群描述了一个用于评估AI模型的新学术基准。
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
arXiv:2604.27865v1 Announce Type: new Abstract: Language models are saturating benchmarks for procedural tasks with narrow objectives. But they are increasingly being deployed in long-horizon, non-stationary environments with open-ended goals. In this paper we introduce KellyBenc…
Language models are saturating benchmarks for procedural tasks with narrow objectives. But they are increasingly being deployed in long-horizon, non-stationary environments with open-ended goals. In this paper we introduce KellyBench, an environment for evaluating sequential deci…