New framework RaRe enhances open-vocabulary scene graph generation using LLM rationales

Researchers have developed RaRe, a novel framework for Open-Vocabulary Scene Graph Generation (OV-SGG). This method leverages multi-modal large language models (MLLMs) to generate chain-of-thought (CoT) rationales, which are then transformed into structured relation embeddings. Unlike previous approaches that used CoT merely as an intermediate step, RaRe directly optimizes these rationale-derived representations to enhance the discriminability between different relations. The framework aligns rationale embeddings with textual relation descriptions via self-supervised contrastive learning and refines rationale generation using a reinforcement learning objective that promotes semantically specific relational evidence. AI

IMPACT This research could improve AI's ability to understand and describe complex visual scenes by enhancing the recognition of relationships between objects, especially rare ones.

RANK_REASON The item is a research paper detailing a new method for scene graph generation. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• alphaXiv
• arXiv
• CatalyzeX
• Connected Papers
• DagsHub
• Gotit.pub
• Hugging Face
• Litmaps
• Open-Vocabulary Scene Graph Generation
• scite Smart Citations
• VG150

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →