Researchers have introduced ScribbleEdit, a new benchmark designed to evaluate the capabilities of Vision-Language Models (VLMs) and Large Language Models (LLMs) in performing image editing tasks based solely on user scribbles. Current models struggle with this type of input, often failing to accurately interpret the user's intent and spatial information from scribbles alone. The ScribbleEdit benchmark includes an automated data construction pipeline and a specific evaluation protocol to measure intention alignment, revealing significant shortcomings in existing VLM/LLM-based editing models. To address these limitations, the researchers propose a soft-token baseline that improves semantic understanding and outperforms standard models on the benchmark. AI
IMPACT This benchmark may drive improvements in interactive image editing tools by focusing VLM/LLM development on understanding user intent from sparse, intuitive inputs.
RANK_REASON The cluster describes a new benchmark and a proposed baseline for a specific AI research problem, published on arXiv.
Read on Hugging Face Daily Papers →
AI-generated summary · Google Gemini · from 2 sources. How we write summaries →
