研究人员开发了一个名为Reconstruction的新基准，用于测试语言模型仅从预发布参考文献中推断研究创意的能力。在六个科学领域和643篇论文的盲测中，七个领先模型仅取得了3-15%的适度成功率。然而，一个结合了跨模型审查和锦标赛选择方法的多代理管道将成功率提高到23-42%，比单一模型性能提升了2.4倍。 AI
影响 该基准可以推动更复杂的LLM在科学文献分析和假设生成方面的能力发展。
排序理由 该集群描述了一个新的学术基准和对语言模型在特定研究任务上的评估，发布在arXiv上。
AI 生成摘要 · Google Gemini · 来自 4 个来源。 我们如何撰写摘要 →
完整方法见我们的编辑标准。
arXiv:2608.16645v1 Announce Type: new Abstract: Can a language model recover the true research idea of a published paper when given only that paper's pre-publication bibliography? We introduce Reconstruction, a blind idea-recovery benchmark that withholds the seed paper and all c…
Can a language model recover the true research idea of a published paper when given only that paper's pre-publication bibliography? We introduce Reconstruction, a blind idea-recovery benchmark that withholds the seed paper and all contemporaneous or future literature, and asks mo…
Can a language model recover the true research idea of a published paper when given only that paper's pre-publication bibliography? We introduce Reconstruction, a blind idea-recovery benchmark that withholds the seed paper and all contemporaneous or future literature, and asks mo…
Can a language model recover the true research idea of a published paper when given only that paper's pre-publication bibliography? We introduce Reconstruction, a blind idea-recovery benchmark that withholds the seed paper and all contemporaneous or future literature, and asks mo…