New method corrects MLLM coordinate prediction bias from positional encoding failures

Researchers have developed a new method called Vision-PE Shuffle Guidance (VPSG) to address inaccuracies in coordinate prediction within multimodal large language models (MLLMs). These models often struggle with precise localization, especially when dealing with high-resolution images where positional encodings can fail and introduce predictable biases. VPSG is a training-free technique applied during inference that mitigates these biases by shuffling positional encodings and using the resulting information to improve digit decoding. Experiments on the ScreenSpot-Pro benchmark demonstrated that VPSG significantly enhances localization accuracy across different model sizes without requiring any retraining. AI

Summary written by gemini-2.5-flash-lite from 1 source. How we write summaries →

IMPACT Improves localization accuracy in MLLMs, potentially enabling more precise applications in vision-language tasks.

RANK_REASON Academic paper introducing a new method for mitigating bias in multimodal large language models.

Read on arXiv cs.CV →

• Xingjian Tao
• ScreenSpot-Pro
• Vision-PE Shuffle Guidance
• arXiv

• paper
• other