Researchers have developed STP-GUI, a novel framework for efficient Web-GUI agents that addresses the issue of excessive visual tokens in high-resolution screenshots. Unlike previous methods that prune tokens independently, STP-GUI employs a query-guided, spatially coherent window-level selection approach. This method preserves fine-grained instruction-conditioned scoring while maintaining local visual structures crucial for recognizing GUI elements. Experiments demonstrate that STP-GUI achieves a favorable efficiency-performance tradeoff, significantly reducing inference latency and FLOPs with minimal performance degradation, and shows promise for transferability to mobile GUI environments. AI
IMPACT This research could lead to more efficient and responsive AI agents for interacting with web and mobile interfaces.
RANK_REASON Academic paper detailing a new method for AI agents. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →