LooseControlVideo: Directorial Video Control using Spatial Blocking
Researchers have developed LooseControlVideo, a novel framework for text-to-video generation that offers intuitive 3D spatial control. Unlike previous methods requiring dense, frame-accurate guidance, LooseControlVideo utilizes sparse, oriented 3D boxes as a proxy for high-level layout and trajectory authoring. The system fine-tunes a Wan 2.2 backbone on a dataset annotated with DNOCS, enabling realistic occlusions and interactions. Evaluations on benchmarks like nuScenes and HO-3D show significant improvements in trajectory accuracy and occlusion handling compared to existing baselines. AI
IMPACT Enhances control and realism in video generation, potentially simplifying complex scene authoring for AI-driven video creation.