New framework enables open-vocabulary dynamic scene graph generation

Researchers have introduced OvDSGG, a novel end-to-end framework for open-vocabulary dynamic scene graph generation. This system addresses the limitations of existing closed-set methods by enabling recognition of objects and predicates beyond a fixed training vocabulary. OvDSGG utilizes an open-vocabulary Spatial Backbone and Temporal Backbone, enhanced by a Triplet Feature Extraction Module and a Visual-Language Alignment Module, which allows for adaptive decision boundaries without costly knowledge distillation. A new benchmark, adapted from Action Genome with disjoint Base/Novel splits, demonstrates OvDSGG's superior performance over existing open-vocabulary baselines. AI

IMPACT This research advances open-vocabulary capabilities in video understanding, potentially improving downstream tasks like video captioning and question answering.

RANK_REASON The cluster contains a research paper detailing a new method for dynamic scene graph generation. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• Action Genome
• arXiv
• Hugging Face
• OvDSGG
• Spatial Backbone
• Temporal Backbone
• Triplet Feature Extraction Module
• Visual-Language Alignment Module

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →