研究人员开发了 MTR-Suite，一个旨在改进对话检索基准的评估和创建的新框架。该套件包括 MTR-Eval，一个基于 LLM 的工具，用于评估现有基准；以及 MTR-Pipeline，一个多代理系统，能以显著降低的成本生成逼真的对话。该框架还引入了 MTR-Bench，一个通用领域基准，模拟了诸如主题切换和冗长等复杂的对话挑战。 AI
影响 引入了一个新框架，用于改进对话检索基准的评估和创建，可能加速 RAG 系统的开发。
排序理由 该集群描述了一篇介绍用于评估和合成对话检索基准的框架的新研究论文。
AI 生成摘要 · Google Gemini · 来自 4 个来源。 我们如何撰写摘要 →
arXiv:2605.25092v1 Announce Type: cross Abstract: Long-term conversational memory is a retrieval workload classical IR was not built for: the index grows during the query stream, query types shift intra-session, and the latency budget per retrieval is sub-10 ms. Lucene-class engi…
Long-term conversational memory is a retrieval workload classical IR was not built for: the index grows during the query stream, query types shift intra-session, and the latency budget per retrieval is sub-10 ms. Lucene-class engines treat the index as static and the query as sta…
Accurate evaluation of conversational retrieval is pivotal for advancing Retrieval-Augmented Generation (RAG) systems. However, existing conversational retrieval benchmarks suffer from costly, sparse human annotation or rigid, unnatural automated heuristics. To address these chal…
Accurate evaluation of conversational retrieval is pivotal for advancing Retrieval-Augmented Generation (RAG) systems. However, existing conversational retrieval benchmarks suffer from costly, sparse human annotation or rigid, unnatural automated heuristics. To address these chal…