研究人员推出了 UI2App，这是一个新的基准测试，旨在评估大型语言模型仅凭屏幕截图推断应用程序行为的能力。该基准测试弥补了现有图像驱动的 LLM 方法的不足，这些方法通常侧重于视觉保真度而非功能交互。UI2App 包含 45 个 Web 应用程序的 327 张屏幕截图，并评估了可执行性、导航、视觉保真度和交互推理。对六个领先的视觉语言模型的实验显示，视觉重建能力与交互实现之间存在显著差异，模型在跨页面状态管理方面尤其困难。 AI
影响 突出了开发能够从视觉输入理解和生成功能性 Web 应用程序的 LLM 的一个关键挑战。
排序理由 该集群包含一篇介绍用于评估 AI 模型的新基准测试的研究论文。
AI 生成摘要 · Google Gemini · 来自 3 个来源。 我们如何撰写摘要 →
arXiv:2607.06306v1 Announce Type: cross Abstract: Large language models (LLMs) have demonstrated growing competence in web page generation. However, existing text-driven approaches rely on complex prompts that impose substantial demands on users and offer limited expressivity for…
Large language models (LLMs) have demonstrated growing competence in web page generation. However, existing text-driven approaches rely on complex prompts that impose substantial demands on users and offer limited expressivity for page layout and cross-page visual coherence. Imag…
Large language models (LLMs) have demonstrated growing competence in web page generation. However, existing text-driven approaches rely on complex prompts that impose substantial demands on users and offer limited expressivity for page layout and cross-page visual coherence. Imag…