研究人员开发了 VideoMDM，一个新颖的基于扩散的框架，用于从 2D 视频监督生成 3D 人体运动。该方法直接从 2D 姿势中训练 3D 运动先验，无需显式的 3D 真实数据。通过使用预训练的 2D 到 3D 提升器作为噪声教师，并采用深度加权 2D 重投影损失，VideoMDM 在 HumanML3D 等基准测试上取得了接近完全 3D 监督模型的性能。该框架还在 Fit3D 和 NBA 等真实视频数据集上取得了成功，生成的运动更受人类评估者的青睐。 AI
影响 通过利用易于获取的 2D 视频数据，为动画和虚拟现实等应用实现更易于访问的 3D 运动生成。
排序理由 这是一篇详细介绍 3D 人体运动生成新方法的学术论文。
在 Hugging Face Daily Papers 阅读 →
AI 生成摘要 · Google Gemini · 来自 3 个来源。 我们如何撰写摘要 →
We introduce VideoMDM, a diffusion-based framework that trains 3D human motion priors directly from accurate 2D poses extracted from monocular videos, without any 3D ground truth. A pretrained 2D-to-3D lifter provides approximate 3D pose sequences that serve as a noisy teacher: t…
VideoMDM trains 3D human motion priors from 2D poses using a diffusion framework with 2D reprojection loss and 3D motion regularizers, achieving near-3D supervised performance without requiring 3D ground truth.
arXiv:2606.13364v1 Announce Type: cross Abstract: We introduce VideoMDM, a diffusion-based framework that trains 3D human motion priors directly from accurate 2D poses extracted from monocular videos, without any 3D ground truth. A pretrained 2D-to-3D lifter provides approximate …