New CAPEval benchmark decouples image caption quality into coverage and precision

Researchers have introduced CAPEval, a new benchmark designed to evaluate image captions by decoupling them into two distinct properties: coverage and precision. Coverage measures how thoroughly a caption describes the visual content, while precision assesses the factual accuracy of the claims made within the caption. Experiments indicate that coverage is a stronger predictor of performance in multimodal understanding tasks, whereas precision is more influential for text-to-image generation tasks. This approach offers a more nuanced assessment of caption quality and provides guidance for optimizing captioners for specific downstream applications. AI

IMPACT Provides a more granular evaluation of image caption quality, aiding in the optimization of captioning models for specific multimodal tasks.

RANK_REASON The item describes a new academic benchmark for evaluating image captions. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• alphaXiv
• arXiv
• CatalyzeX
• CORE Recommender
• DagsHub
• Gotit.pub
• Hugging Face
• Influence Flower
• ScienceCast

• paper
• product

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →