研究人员推出了 PredicateLongBench，这是一个新的基准，旨在通过测试大型语言模型识别满足特定谓词的最长连续单词子序列的能力来评估其长上下文能力。与通常衡量平均性能的现有基准不同，PredicateLongBench 系统地探索了各种难度轴来探测模型的局限性。该基准利用合成和真实世界的数据管道，揭示了当前前沿模型在这些更具挑战性的长上下文任务中存在困难。 AI
影响 该基准通过突出特定的薄弱环节，有望推动 LLM 长上下文理解能力的提升。
排序理由 该集群包含一篇介绍用于评估 LLM 能力的新基准的学术论文。
AI 生成摘要 · Google Gemini · 来自 3 个来源。 我们如何撰写摘要 →
arXiv:2607.08284v1 Announce Type: new Abstract: Large language models (LLMs) have demonstrated rapidly improving long-context capabilities, prompting a wave of benchmarks designed to evaluate them. However, existing long-context evaluations - from Needle-in-a-Haystack (NIAH) test…
Large language models (LLMs) have demonstrated rapidly improving long-context capabilities, prompting a wave of benchmarks designed to evaluate them. However, existing long-context evaluations - from Needle-in-a-Haystack (NIAH) tests to more recent multi-hop reasoning and summari…
Large language models (LLMs) have demonstrated rapidly improving long-context capabilities, prompting a wave of benchmarks designed to evaluate them. However, existing long-context evaluations - from Needle-in-a-Haystack (NIAH) tests to more recent multi-hop reasoning and summari…