Perplexity 已开源 WANDR，这是一个内部基准，旨在评估 AI 代理的深度和广度研究能力。WANDR 包含 500 项研究任务，需要跨三个难度级别超过 170,000 条来源支持的记录。WANDR 不依赖于黄金解决方案，而是重新抓取引用的页面，以根据底层证据验证声明，从而实现时变事实并提供密集、可解释的评估信号。 AI
影响 为 AI 代理提供了一个新的、基于证据的评估框架，有可能推动事实核查和研究能力的改进。
排序理由 用于 AI 代理评估的内部基准的开源。
AI 生成摘要 · Google Gemini · 来自 7 个来源。 我们如何撰写摘要 →
Soft scores give partial credit. Hard scores require every component for a member to be complete and correct. The benchmark tasks and evaluation harness are available at: https://t.co/o2HX0a4RTQ
Instead of grading against a gold solution, WANDR re-fetches every cited page and checks each claim against the underlying evidence. This allows certain tasks to contain time-varying facts. https://t.co/2sDRQrSoSx
WANDR is constructed from de-identified production use cases covering people’s day-to-day research tasks like competitive research, due diligence, literature review, market analysis, product comparison, talent sourcing, and more. https://t.co/y6dClkm2a6
WANDR tests how well agents discover large sets of entities and verify specific facts about each one. It provides a dense, interpretable eval signal that reveals whether an agent fails, and where. The pipeline also doubles as a semi-automated factory for training data. https://t…
WANDR represents these requirements as hierarchical, independently verifiable records. It consists of 500 research tasks that require 170,495 source-backed records across three tiers of difficulty. https://t.co/OX5e1zuGbe
Wide-and-deep research requires two capabilities. 1) Agents must search broadly enough to find all qualifying entities 2) Agents must investigate deep enough to support every claim with evidence. https://t.co/l9GE3z3FvW
We’re open sourcing WANDR. WANDR is an internal benchmark we built and used for building deep and wide research capabilities inside Perplexity Computer. https://t.co/gp2BWjFK4d