New LookBack method scores LVLM responses by visual grounding

Researchers have introduced LookBack, a novel method for scoring responses from Large Vision-Language Models (LVLMs). Existing methods, which rely on text-based confidence scores, are insufficient for LVLMs as they do not adequately assess the model's grounding in visual input. LookBack addresses this by incorporating a visual lookback score, which measures the token-level reference to image content, alongside traditional token likelihood. This approach has demonstrated consistent improvements in selecting the best response across multiple benchmarks and models with minimal added computational cost. AI

IMPACT Improves evaluation of multimodal models, potentially leading to more reliable and grounded AI systems.

RANK_REASON Academic paper detailing a new method for evaluating LVLM responses. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• arXiv
• Large Vision Language Models
• LVLMs

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →