New framework ReferGaussian enables language-based segmentation in dynamic 4D scenes

Researchers have introduced R4DGS, a new task and framework for referring segmentation in dynamic 4D scenes using Gaussian Splatting. This approach, called ReferGaussian, converts low-level dynamic Gaussians into language-addressable entities. It utilizes multimodal large language models to break down complex queries into trackable object phrases and structured referring constraints, enabling constraint-aware grounding without scene-specific semantic optimization. Experiments on the R4D-Bench-QA benchmark and the 4D LangSplat protocol show improved temporal target selection and spatiotemporal grounding. AI

IMPACT Enhances the ability to ground natural language queries in complex, dynamic 3D environments, potentially improving applications in robotics and augmented reality.

RANK_REASON The item describes a new research paper detailing a novel task, benchmark, and framework for referring segmentation in dynamic 4D scenes. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• 4D Gaussian Splatting
• 4D LangSplat
• arXiv
• EntityBank
• Multimodal Large Language Models
• R4D-Bench-QA
• R4DGS
• ReferGaussian

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →