研究人员推出StakeBench，一个旨在通过基于市场承诺而非主观人类标签来评估大型语言模型（LLM）语言理解能力的新评估框架。该框架利用了来自Polymarket和Manifold等平台已解决市场的超过56万条评论，将其与可观察到的交易行为和市场赔率联系起来。对15个LLM的初步评估显示，虽然模型可以部分恢复仓位信号，但它们在预测未来行动或进行集体赔率预测等更复杂的任务上遇到困难，模型规模和金融领域调优与性能的相关性很小。 AI
影响 引入了一种新颖的LLM评估方法，侧重于市场承诺信号而非主观情绪，可能带来更强大的金融NLP应用。
排序理由 该集群包含一篇介绍LLM新评估框架的研究论文。
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
arXiv:2605.26074v1 Announce Type: cross Abstract: Existing financial NLP benchmarks often rely on labels supplied by outside observers, measuring how language is perceived rather than what speakers have committed to in the market. We introduce StakeBench, an evaluation framework …
Existing financial NLP benchmarks often rely on labels supplied by outside observers, measuring how language is perceived rather than what speakers have committed to in the market. We introduce StakeBench, an evaluation framework for language understanding grounded in market comm…