OmniDirector: General Multi-Shot Camera Cloning without Cross-Paired Data
Researchers have developed OmniDirector, a novel framework for cloning camera motion in video generation. This system utilizes a general camera motion representation that encodes cameras as grid motion videos, overcoming limitations of existing methods that struggle with multi-shot generation or data scarcity. OmniDirector is trained on a large dataset of camera grid-video pairs and offers director-level control for multimodal diffusion transformers by coordinating characters, actions, and cameras. AI