New RMOT task and TempRMOT framework advance multi-object tracking with language

Researchers have introduced a new task called referring multi-object tracking (RMOT) to improve the localization of objects described in natural language expressions. This task addresses limitations in current methods by enabling the modeling of object dynamics in terms of quantity and temporal states. To support RMOT, a new benchmark named Refer-KITTI-V2 was created, featuring diverse language expressions and a semi-automatic labeling pipeline that generated over 9,700 language prompts. The proposed TempRMOT framework, an end-to-end Transformer-based model, utilizes a query-driven Temporal Enhancement Module to refine object representations and has demonstrated state-of-the-art performance on existing and new benchmarks. AI

IMPACT This research advances multi-object tracking capabilities by integrating natural language understanding, potentially improving applications that require precise object localization based on descriptive text.

RANK_REASON The cluster contains an academic paper detailing a new task, benchmark, and framework for referring multi-object tracking. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• arXiv
• Refer-KITTI
• Refer-KITTI-V2
• TempRMOT
• transformer
• Yani Zhang

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →