AtelierEval: Agentic Evaluation of Humans & LLMs as Text-to-Image Prompters
Researchers have introduced AtelierEval, a novel benchmark designed to evaluate the proficiency of both humans and multimodal large language models (MLLMs) in generating effective text-to-image prompts. This benchmark, which includes 360 expert-crafted tasks, aims to quantify the quality of prompts used to translate user intent into detailed instructions for text-to-image systems. AtelierEval also features AtelierJudge, an agentic evaluator that correlates strongly with human expert assessments, and its experiments reveal that mimicry-based prompting may be more effective than planning-based approaches for future prompters. AI
IMPACT Introduces a new evaluation framework for text-to-image prompting, enabling better assessment of both human and AI prompter capabilities.