研究人员引入了HieraRAG，一个通过分析问题粒度来评估检索增强生成（RAG）系统的分层框架。该框架旨在帮助实践者确定RAG基准测试的最佳详细程度，以最大化其区分能力。一项案例研究生成了超过5000个合成问答对，揭示了最佳粒度因维度而异，复杂性受益于细粒度区分，而其他方面在中间粒度达到峰值。此外，还开发了一种新指标——连贯性比率（Coherence Ratio），用于评估细粒度分割如何很好地细分父类别。 AI
影响 这些新框架和基准测试为LLM和RAG系统提供了更细致的评估方法，有望带来更强大、更具能力的AI应用。
排序理由 该集群包含多篇介绍LLM和RAG系统评估新框架和基准测试的学术论文。
AI 生成摘要 · Google Gemini · 来自 4 个来源。 我们如何撰写摘要 →
arXiv:2606.12789v1 Announce Type: new Abstract: Evaluating retrieval-augmented generation (RAG) systems requires benchmarks that capture diverse question characteristics, yet practitioners lack empirical guidance on which dimensions to vary and at what granularity. We present Hie…
Evaluating retrieval-augmented generation (RAG) systems requires benchmarks that capture diverse question characteristics, yet practitioners lack empirical guidance on which dimensions to vary and at what granularity. We present HieraRAG, a hierarchical framework for studying gra…
arXiv:2602.12424v2 Announce Type: replace-cross Abstract: Benchmarks establish a standardized evaluation framework to systematically assess the performance of large language models (LLMs), facilitating objective comparisons and driving advancements in the field. However, existing…
arXiv:2411.19504v2 Announce Type: replace Abstract: The advance of large language models (LLMs) has unlocked great opportunities in complex multi-modal data management tasks, particularly in question answering (QA) over complicated multi-table relational data. Despite significant…