PulseAugur
实时 14:33:52
English(EN) UniHEAR: Unified Heterogeneous-Source Attentive Retrieval for Knowledge-Based Visual Question Answering

新方法增强多模态模型用于视觉问答 · 跟踪 6 个来源

研究人员开发了几种新方法来提高多模态大语言模型 (MLLMs) 在知识驱动视觉问答 (KB-VQA) 中的性能。一种名为“Look Twice”的方法是一个无需训练的框架,它利用模型的内部注意力来突出相关的视觉和文本证据,在各种基准测试和 MLLMs 上将准确率提高了多达 12.5 个百分点。另一种方法是贝叶斯数据重加权,通过在训练过程中自适应地降低潜在不相关负例的权重来增强多模态检索。此外,UniHEAR 提供了一个统一的框架,用于异构源实体检索和重排序,解决了单模态检索的局限性并提高了召回率。 AI

影响 这些进展可能带来更准确、更高效的 AI 系统,能够理解和响应复杂的视觉和文本信息。

排序理由 多篇 arXiv 论文介绍了多模态 AI 任务的新方法。

在 arXiv cs.IR (Information Retrieval) 阅读 →

AI 生成摘要 · Google Gemini · 来自 6 个来源。 我们如何撰写摘要 →

新方法增强多模态模型用于视觉问答 · 跟踪 6 个来源

报道来源 [6]

  1. arXiv cs.AI TIER_1 English(EN) · Marco Morini, Sara Sarto, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara ·

    再看一遍:面向知识型视觉问答的无训练证据高亮

    arXiv:2604.01280v2 Announce Type: replace-cross Abstract: Knowledge-based Visual Question Answering (KB-VQA) requires Multimodal Large Language Models (MLLMs) to identify and combine fine-grained visual cues with retrieved textual evidence. However, retrieval often introduces noi…

  2. arXiv cs.LG TIER_1 English(EN) · Jingchen Sun, Shaobo Han, Ruiyi Zhang, Naresh Kumar Devulapally, Ming Liu, Yitao Long, Vishnu Suresh Lokhande, Changyou Chen ·

    贝叶斯数据重加权改进基于知识的视觉问答的多模态检索

    arXiv:2608.02907v1 Announce Type: new Abstract: Multimodal retrievers are essential for knowledge-based visual question answering, where they retrieve external evidence for image-question pairs. However, existing contrastive training methods typically treat all unmatched query-do…

  3. arXiv cs.CL TIER_1 English(EN) · Ganzhong Luo, Yang Ren, Hanyong Wang, Shuyu Zheng, Menglong Yang ·

    UniHEAR:统一的异构源注意力检索用于知识驱动的视觉问答

    arXiv:2608.01147v1 Announce Type: cross Abstract: Knowledge-Based Visual Question Answering (KB-VQA) requires retrieving relevant entity knowledge from external sources to answer visually grounded questions. Existing retrieval-augmented systems suffer from two critical limitation…

  4. arXiv cs.IR (Information Retrieval) TIER_1 English(EN) · Menglong Yang ·

    UniHEAR:统一的异构源注意力检索用于知识驱动的视觉问答

    Knowledge-Based Visual Question Answering (KB-VQA) requires retrieving relevant entity knowledge from external sources to answer visually grounded questions. Existing retrieval-augmented systems suffer from two critical limitations. First, relying on a single retrieval modality c…

  5. arXiv cs.IR (Information Retrieval) TIER_1 English(EN) · Menglong Yang ·

    UniHEAR:统一的异构源注意力检索用于知识驱动的视觉问答

    Knowledge-Based Visual Question Answering (KB-VQA) requires retrieving entity knowledge from external sources to answer visually grounded questions. Existing retrieval-augmented systems suffer from two critical limitations. First, relying on a single retrieval modality creates a …

  6. arXiv cs.CV TIER_1 English(EN) · Haokun Wen, Xuemeng Song, Haoyu Zhang, Weili Guan, Xiangyu Zhao, Liqiang Nie ·

    UniCVR:从对齐到重排,实现统一的零样本组合视觉检索

    arXiv:2604.20318v2 Announce Type: replace Abstract: Composed image retrieval, multi-turn composed image retrieval, and composed video retrieval all share a common paradigm: composing the reference visual with modification text to retrieve the desired target. Despite this shared s…