New framework ReCap improves zero-shot image captioning by realigning entities

Researchers have introduced ReCap, a novel framework designed to improve zero-shot image captioning by refining synthetic image-text pairs. Unlike previous methods that focused on global similarity, ReCap addresses fine-grained entity-level misalignment by using detected image entities to guide caption rewriting. This approach enhances the fidelity of synthetic supervision and can be integrated into existing data pipelines. Experiments demonstrate that ReCap consistently improves image-text consistency and achieves state-of-the-art results on various zero-shot image captioning benchmarks. AI

IMPACT Enhances the accuracy and faithfulness of AI-generated image descriptions by improving synthetic data quality.

RANK_REASON Academic paper introducing a new framework for image captioning. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• alphaXiv
• arXiv
• CatalyzeX
• Connected Papers
• CORE Recommender
• DagsHub
• Gotit.pub
• Hugging Face
• Litmaps
• ReCap
• ScienceCast
• scite Smart Citations

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →