ReferFlow introduces trajectory-level REC for multimodal LLMs

Researchers have introduced ReferFlow, a novel approach designed to improve how multimodal large language models handle complex referring expressions in computer vision tasks. Unlike previous methods that focus solely on the final target, ReferFlow addresses trajectory-level referring expression comprehension by evaluating a model's ability to localize intermediate visual references across a sequence of grounding steps. The accompanying ReferFlow-Bench benchmark was created to test this capability, revealing that current models struggle with consistent intermediate grounding. ReferFlow itself decouples trajectory generation from coordinate decoding, employing Stateful Region Latent Reasoning and Trajectory-Aware Parallel Decoding to preserve region information and refine coordinates efficiently. AI

IMPACT Enhances multimodal LLM capabilities in complex visual grounding tasks, potentially improving their understanding of sequential instructions.

RANK_REASON This is a research paper detailing a new method and benchmark for referring expression comprehension in computer vision. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• arXiv
• Grounding-Aware GRPO
• Hugging Face
• ReferFlow
• ReferFlow-Bench
• Stateful Region Latent Reasoning
• Trajectory-Aware Parallel Decoding

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →