研究人员开发了DREAM（Dense Retrieval Embeddings via Autoregressive Modeling），一种新颖的训练密集检索系统的方法。与依赖昂贵标注数据的传统方法不同，DREAM利用大型语言模型（LLMs）的下一个词预测目标来监督训练过程。通过将查询-文档相似度分数注入LLM的注意力头，DREAM使预测损失能够为检索器提供梯度。在检索基准上的评估表明，DREAM在各种模型规模下始终优于现有基线。 AI
影响 这种方法可以减少训练检索系统对昂贵标注数据集的依赖，从而可能加速开发。
排序理由 该集群描述了一篇详细介绍新AI模型训练方法的论文。
在 Hugging Face Daily Papers 阅读 →
AI 生成摘要 · Google Gemini · 来自 3 个来源。 我们如何撰写摘要 →
arXiv:2606.24667v1 Announce Type: new Abstract: Dense retrieval embedding models are a fundamental component of modern retrieval-based AI systems. Most dense retrievers are trained with contrastive objectives, which require labeled positive and negative document pairs that are of…
Dense retrieval embedding models are a fundamental component of modern retrieval-based AI systems. Most dense retrievers are trained with contrastive objectives, which require labeled positive and negative document pairs that are often costly and difficult to obtain. In this work…
DREAM trains dense retrieval embeddings using autoregressive language model attention mechanisms to supervise document-query similarity without requiring labeled examples.