New DAR method enables video models to render 4D scenes

Researchers have developed DAR, a novel approach that enables pretrained video diffusion models to function as 4D renderers. This method conditions these models using an animated mesh, camera trajectory, and a reference image, allowing for scene generation that accounts for both camera movement and internal object animation. DAR projects a neural 4D G-buffer, incorporating tracking, world position, and normal information, which is then integrated into the model via a widened control adapter. Evaluations on the DAR-4D benchmark demonstrate significant improvements in PSNR, SSIM, and LPIPS compared to existing methods like Wan2.2-Depth. AI

IMPACT This research could enable more sophisticated 4D content generation and manipulation by leveraging existing video diffusion models.

RANK_REASON The item is a research paper detailing a new method for 4D rendering using video models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• alphaXiv
• arXiv
• camera trajectory
• CatalyzeX
• DagsHub
• DAR-4D benchmark
• Gotit.pub
• Hugging Face
• ScienceCast
• video diffusion models
• Wan2.2
• Wan2.2-Depth

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →