CogCanvas: A Benchmark for Evaluating Multi-Subject Reference-Based Image Generation
Researchers have introduced CogCanvas, a new benchmark designed to evaluate the capabilities of image generation models in complex multi-subject scenarios. This benchmark addresses limitations in existing tools by assessing the joint preservation of multiple identities, object binding, and background consistency. CogCanvas includes 1,952 curated images and 1,361 compositional prompts, supporting tasks like reference-based multi-human-object generation and text-to-image generation. Initial benchmarking of state-of-the-art models revealed significant degradation in performance as the number of subjects increased, particularly in binding objects and fashion items. AI
IMPACT This benchmark highlights current limitations in AI image generation for complex scenes, potentially guiding future research towards more robust multi-subject and compositional capabilities.