English(EN) Do LLM Attribution Metrics Transfer? Auditing Retrieval-Augmented Generation Evaluation Across Datasets and Constructs

研究发现LLM归因指标缺乏跨数据集的可迁移性

作者 PulseAugur 编辑部 · [2 个来源] · 2026-06-22 20:25

一篇新的研究论文调查了用于评估检索增强生成（RAG）系统归因的自动指标的可靠性。研究发现，包括词汇、嵌入和BERTScore基线在内的常用归因指标在不同数据集和评估构造上表现不一致。指标排名可能显著反转，导致具体的决策成本，即基于平均性能选择指标可能比固定一个评分器更差。虽然LLM裁判提供了替代方案，但它们成本更高且不确定，将验证负担转移了，而不是消除了它。 AI

影响强调了在RAG系统中对归因指标进行特定数据集验证的必要性，影响了如何可靠地评估LLM输出。

排序理由该集群包含一篇详细介绍LLM评估指标研究结果的学术论文。

在 arXiv cs.IR (Information Retrieval) 阅读 →

AI 生成摘要 · Google Gemini · 来自 2 个来源。我们如何撰写摘要 →

报道来源 [2]

arXiv cs.CL TIER_1 English(EN) · Tianyu Ding, Aditya Nannapaneni, Juan Pablo De la Cruz Weinstein · 2026-06-24 04:00

Do LLM Attribution Metrics Transfer? Auditing Retrieval-Augmented Generation Evaluation Across Datasets and Constructs

arXiv:2606.23915v1 Announce Type: new Abstract: Practice often treats automatic metrics for attribution in LLM retrieval-augmented generation as interchangeable. We audit eight automatic scorers -- lexical, embedding, and BERTScore baselines alongside entailment/grounding-trained…
arXiv cs.IR (Information Retrieval) TIER_1 English(EN) · Juan Pablo De la Cruz Weinstein · 2026-06-22 20:25

Do LLM Attribution Metrics Transfer? Auditing Retrieval-Augmented Generation Evaluation Across Datasets and Constructs

Practice often treats automatic metrics for attribution in LLM retrieval-augmented generation as interchangeable. We audit eight automatic scorers -- lexical, embedding, and BERTScore baselines alongside entailment/grounding-trained models (clean and FEVER NLI, the checker MiniCh…

报道来源 [2]

Do LLM Attribution Metrics Transfer? Auditing Retrieval-Augmented Generation Evaluation Across Datasets and Constructs

Do LLM Attribution Metrics Transfer? Auditing Retrieval-Augmented Generation Evaluation Across Datasets and Constructs

相关实体

相关话题