UniCaMo framework enhances video generation with 3D-grounded motion control

Researchers have introduced UniCaMo, a novel framework designed to enhance controllability in video generation models. This system allows for simultaneous control over both object motion and camera viewpoint by directly manipulating the initial noise input to diffusion models. UniCaMo constructs a shared 3D-grounded motion-consistent noise space, utilizing sparse 3D point tracks for object trajectory guidance and a spherical noise representation for camera motion consistency. The framework integrates seamlessly with existing video diffusion models through lightweight fine-tuning, achieving state-of-the-art results in controllable video generation. AI

IMPACT Enhances controllability in video generation, potentially leading to more sophisticated AI-powered content creation tools.

RANK_REASON The cluster contains a research paper detailing a new framework for video generation. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• alphaXiv
• arXiv
• DagsHub
• Gotit.pub
• Hugging Face
• Lora
• ScienceCast
• UniCaMo
• WAN 2.1

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →