研究人员开发了InnerZoom，一个新颖的框架，用于在单次前向传播中实现准确高效的GUI基础定位。该方法通过在解码器层之间保留目标区域感知来解决现有多模态大语言模型（MLLM）方法的局限性，这对于GUI交互中精确坐标的生成至关重要。InnerZoom在多个基准测试中取得了最先进的性能，在提高精度的同时降低了计算成本和延迟。 AI
影响 这种新方法可以提高AI代理与图形用户界面交互的效率和准确性。
排序理由 该集群报道了一篇详细介绍一种新GUI基础定位方法的最新研究论文。
在 Hugging Face Daily Papers 阅读 →
AI 生成摘要 · Google Gemini · 来自 4 个来源。 我们如何撰写摘要 →
完整方法见我们的编辑标准。
arXiv:2604.14262v2 Announce Type: replace-cross Abstract: GUI grounding models report over 85% accuracy on standard benchmarks, yet drop 27-56 percentage points when instructions require spatial reasoning rather than direct element naming. Current benchmarks miss this because the…
InnerZoom addresses GUI grounding challenges by preserving target-region awareness across decoder layers through a single-forward pass that bridges cross-layer evidence, achieving state-of-the-art performance with reduced computational cost.
arXiv:2606.30084v1 Announce Type: new Abstract: MLLM-based GUI grounding methods commonly formulate target localization as autoregressive coordinate generation, enabling models to leverage the strong instruction-following and semantic understanding capabilities of MLLMs. However,…
MLLM-based GUI grounding methods commonly formulate target localization as autoregressive coordinate generation, enabling models to leverage the strong instruction-following and semantic understanding capabilities of MLLMs. However, this formulation requires the model to retain r…