ProxyUp framework enables training-free controllable video generation

Researchers have developed ProxyUp, a novel framework for training-free video generation that uses a proxy video to control complex dynamics. This method allows for precise control over foreground object motion and interactions, even when paired proxy-target videos are unavailable. ProxyUp achieves this by inverting the proxy video into a latent representation, applying region-wise latent noising to preserve motion-critical elements while allowing for text-driven regeneration, and using Stochastic Flow Relaxation (SFR) to address distribution mismatches. AI

IMPACT Enables more precise control over video generation dynamics, potentially improving realism and user control in applications like animation and content creation.

RANK_REASON The cluster contains a research paper detailing a new method for video generation. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• arXiv
• Hugging Face
• ProxyUp
• region-wise latent noising
• Stochastic Flow Relaxation (SFR)

• paper
• product

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →