PulseAugur
EN
LIVE 14:35:14

ProMSA agent advances knowledge-based visual question answering

By PulseAugur Editorial · [3 sources] ·

Researchers have developed ProMSA, a novel agent designed for knowledge-based visual question answering (KB-VQA). Unlike previous methods that use fixed retrieval pipelines, ProMSA adaptively selects between image search, text search, or stopping based on tool-call budgets and deduplication. The agent is trained using a combination of rejection-sampling SFT and a sequence-level RL objective called TN-GSPO. Experiments on E-VQA and InfoSeek datasets demonstrate that ProMSA achieves improved retrieval and end-to-end accuracy compared to existing RAG and agent baselines. AI

IMPACT Advances agent-based reasoning for multimodal tasks, potentially improving complex information retrieval systems.

RANK_REASON Publication of a new research paper detailing a novel AI agent and its methodology.

Read on arXiv cs.AI →

AI-generated summary · Google Gemini · from 3 sources. How we write summaries →

ProMSA agent advances knowledge-based visual question answering

COVERAGE [3]

  1. arXiv cs.AI TIER_1 English(EN) · ZhengXian Wu, Hangrui Xu, Kai Shi, Zhuohong Chen, Yunyao Yu, Chuanrui Zhang, Zirui Liao, Jun Yang, Zhenyu Yang, Haonan Lu, Haoqian Wang ·

    ProMSA:Progressive Multimodal Search Agents for Knowledge-Based Visual Question Answering

    arXiv:2606.27974v1 Announce Type: cross Abstract: Knowledge-based Visual Question Answering (KB-VQA) requires models to combine image understanding with external knowledge. Most prior methods use a fixed retrieve-then-generate pipeline with a pre-selected retriever and a static t…

  2. arXiv cs.AI TIER_1 English(EN) · Haoqian Wang ·

    ProMSA:Progressive Multimodal Search Agents for Knowledge-Based Visual Question Answering

    Knowledge-based Visual Question Answering (KB-VQA) requires models to combine image understanding with external knowledge. Most prior methods use a fixed retrieve-then-generate pipeline with a pre-selected retriever and a static top-k setting, which is not adaptive during reasoni…

  3. Hugging Face Daily Papers TIER_1 English(EN) ·

    ProMSA:Progressive Multimodal Search Agents for Knowledge-Based Visual Question Answering

    A progressive multimodal search agent for knowledge-based visual question answering that adaptively selects search strategies and optimizes through sequence-level reinforcement learning.

RELATED ENTITIES

RELATED TOPICS