PulseAugur
实时 07:30:36
English(EN) The Surprising Effectiveness of Video Diffusion Models for Hand Motion Reconstruction

新研究探索使用扩散模型进行先进的视频生成和处理

研究人员正在探索使用扩散模型来改进视频生成和处理的先进技术。一种方法是将状态空间模型(SSM)与视频扩散模型集成,以提高效率并处理更长的序列,在内存使用和性能方面优于基于注意力的方法。其他研究则侧重于通过使用扩散 Transformer 和自适应来提高视频重新照明中的时间一致性,并通过利用预训练的视频扩散模型从视频中重建 4D 手部运动。此外,还在开发用于高效视频恢复和鲁棒点跟踪的方法,方法是调整扩散模型的特征和训练策略。 AI

影响 视频扩散模型的进步有望实现更高效、更连贯的视频生成、改进的重新照明以及对手部运动等复杂运动的更好重建。

排序理由 多篇研究论文详细介绍了使用扩散模型及相关架构在视频生成、恢复和跟踪方面的新颖方法和改进。

在 Hugging Face Daily Papers 阅读 →

AI 生成摘要 · Google Gemini · 来自 23 个来源。 我们如何撰写摘要 →

新研究探索使用扩散模型进行先进的视频生成和处理

报道来源 [23]

  1. arXiv cs.AI TIER_1 Italiano(IT) · Donghyun Lee, Jitesh Chavan, Duy Nguyen, Sam Huang, Liming Jiang, Priyadarshini Panda, Timo Mertens, Saurabh Shukla ·

    OrbitQuant:图像和视频扩散 Transformer 的数据无关量化

    arXiv:2607.02461v1 Announce Type: cross Abstract: Diffusion transformers (DiTs) achieve state-of-the-art image and video generation, but their multi-step sampling and growing parameter count make inference expensive. Post-training quantization (PTQ) is the natural remedy, yet DiT…

  2. arXiv cs.AI TIER_1 Italiano(IT) · Saurabh Shukla ·

    OrbitQuant:图像和视频扩散 Transformer 的数据无关量化

    Diffusion transformers (DiTs) achieve state-of-the-art image and video generation, but their multi-step sampling and growing parameter count make inference expensive. Post-training quantization (PTQ) is the natural remedy, yet DiT activations shift across timesteps, prompts, and …

  3. arXiv cs.LG TIER_1 English(EN) · Yuxi Liu, Yipeng Hu, Zekun Zhang, Kunze Jiang, Kun Yuan ·

    分布混合很重要:用于高效视频扩散 Transformer 的动态稀疏注意力

    arXiv:2601.11641v3 Announce Type: replace-cross Abstract: While Diffusion Transformers (DiTs) have achieved notable progress in video generation, this long-sequence generation task remains constrained by the quadratic complexity inherent to self-attention mechanisms, creating sig…

  4. arXiv cs.AI TIER_1 English(EN) · Yujin Tang, Tian Zhou, Xin Lin, Cheng Tan, Yifan Hu, Rong Jin, SouYoung Jin, Liang Sun ·

    利用预测性可微分渲染学习视频动态

    arXiv:2606.31050v1 Announce Type: cross Abstract: How to accurately predict a high-fidelity future world? While the visual world is inherently continuous, existing deterministic video prediction models operate in discrete pixel space and are mainly optimized with pixel-wise mean …

  5. arXiv cs.AI TIER_1 English(EN) · Yuta Oshima, Shohei Taniguchi, Masahiro Suzuki, Yutaka Matsuo ·

    SSM 遇见视频扩散模型:通过结构化状态空间实现高效的长期视频生成

    arXiv:2403.07711v5 Announce Type: replace-cross Abstract: Given the remarkable achievements in image generation through diffusion models, the research community has shown increasing interest in extending these models to video generation. Recent diffusion models for video generati…

  6. arXiv cs.LG TIER_1 English(EN) · Jing Yang, Mayoore Jaiswal, Zian Wang, Steven Zeng, Rochelle Pereira, Yajie Zhao, Jianyuan Min ·

    HorizonRelight:通过Diffusion Transformers实现长视域视频的一致性重光照

    arXiv:2606.29095v1 Announce Type: cross Abstract: Diffusion-based video relighting enables controllable relighting from a single input video, but modern video diffusion backbones are trained on short clips and applied to long-horizon videos through chunked sliding-window inferenc…

  7. Hugging Face Daily Papers TIER_1 English(EN) ·

    视频扩散模型在手部动作重建方面的惊人有效性

    ViDiHand uses pretrained video diffusion model representations with hand-overlay rendering to reconstruct 4D hand motion directly from video frames without detectors or optimization.

  8. arXiv cs.CV TIER_1 English(EN) · Kyobin Choo, Youngmin Kim, Hyunkyung Han, Geunrip Park, Chanyoung Kim, Sunyoung Jung, Seong Jae Hwang ·

    QWERTY:通过查询扭曲视频扩散 Transformer 实现的无训练运动控制

    arXiv:2607.01869v1 Announce Type: new Abstract: Video diffusion transformers (DiTs) generate high-fidelity and temporally coherent videos, yet motion control remains implicit, primarily relying on text prompts. As a result, achieving desired motion often requires extensive prompt…

  9. arXiv cs.CV TIER_1 English(EN) · Xuanhua He, Jiaxin Xie, Mingzhe Zheng, Qifeng Chen ·

    ICDepth:通过上下文条件化驯服视频扩散模型用于视频深度估计

    arXiv:2607.01677v1 Announce Type: new Abstract: Monocular video depth estimation requires temporal consistency, geometric accuracy, and generalization across diverse scenarios, yet existing methods struggle to achieve all three simultaneously. Discriminative models excel at per-f…

  10. arXiv cs.CV TIER_1 English(EN) · Edoardo A. Dominici, Thomas Deixelberger, Konstantinos Vardis, Markus Steinberger ·

    Control-DINO:可控图像到视频扩散的特征空间条件化

    arXiv:2604.01761v2 Announce Type: replace Abstract: Video diffusion models have recently been applied with success to problems in content generation, novel view synthesis, and, more broadly, world simulation. Many applications in generation and transfer rely on conditioning these…

  11. arXiv cs.CV TIER_1 English(EN) · Yunuo Chen, Chuqin Zhou, Jiangchuan Li, Xiaoyue Ling, Bing He, Jincheng Dai, Li Song, Guo Lu ·

    利用视频扩散先验进行超低比特率图像压缩的下一帧解码

    arXiv:2603.15129v3 Announce Type: replace Abstract: We present a novel paradigm for ultra-low-bitrate image compression (ULB-IC) that exploits the ``temporal'' evolution in generative image compression. Specifically, we define an explicit intermediate state during decoding: a com…

  12. arXiv cs.CV TIER_1 English(EN) · Geunhyuk Youk, Jeonghyeok Do, Dayeon Kim, Jihyong Oh, Munchurl Kim ·

    AVSR-Diff:用于时间一致任意尺度视频超分辨率的尺度无关扩散先验

    arXiv:2607.00987v1 Announce Type: new Abstract: Diffusion models have significantly advanced video super-resolution (VSR) but remain largely constrained to fixed upsampling scales. Conversely, while coordinate-based arbitrary-scale VSR methods offer scale flexibility, they inhere…

  13. arXiv cs.CV TIER_1 English(EN) · Yuwei Guo, Ceyuan Yang, Hao He, Yang Zhao, Meng Wei, Zhenheng Yang, Weilin Huang, Dahua Lin ·

    通过自重采样实现自回归视频扩散的端到端训练

    arXiv:2512.15702v2 Announce Type: replace Abstract: Autoregressive video diffusion models hold promise for world simulation but are vulnerable to exposure bias arising from the train-test mismatch. While recent works address this via post-training, they typically rely on a bidire…

  14. arXiv cs.CV TIER_1 English(EN) · Munchurl Kim ·

    AVSR-Diff:用于时间一致任意尺度视频超分辨率的尺度无关扩散先验

    Diffusion models have significantly advanced video super-resolution (VSR) but remain largely constrained to fixed upsampling scales. Conversely, while coordinate-based arbitrary-scale VSR methods offer scale flexibility, they inherently suffer from severe over-smoothing at large …

  15. arXiv cs.CV TIER_1 English(EN) · Yuxi Wang, Chengkai Jin, Yufei Liu, Wenqi Ouyang, Tianyi Wei, Zhiwei Zeng, Siyuan Huang, Zhiqi Shen, Xingang Pan ·

    视频扩散模型在手部运动重建方面的惊人有效性

    arXiv:2606.30308v1 Announce Type: new Abstract: 4D hand motion reconstruction from egocentric video is bottlenecked by clear limitations of existing methods: image-based pipelines depend on a detector that fails under heavy occlusion, while video-based methods rely on temporal mo…

  16. arXiv cs.CV TIER_1 English(EN) · Haoran Bai, Xiaoxu Chen, Canqian Yang, Zongyao He, Sibin Deng, Ying Chen ·

    Vivid-VR:从文本到视频的扩散 Transformer 中提炼概念以实现照片级视频修复

    arXiv:2508.14483v4 Announce Type: replace Abstract: We present Vivid-VR, a DiT-based generative video restoration method built upon an advanced T2V foundation model, where ControlNet is leveraged to control the generation process, ensuring content consistency. However, convention…

  17. arXiv cs.CV TIER_1 English(EN) · Soowon Son, Honggyu An, Jisu Nam, Hyunah Ko, Chaehyun Kim, Dahyun Chung, Siyoon Jin, Jung Yi, Junhwa Hur, Seungryong Kim ·

    探索和利用视频扩散Transformer特征以实现鲁棒点跟踪

    arXiv:2512.20606v2 Announce Type: replace Abstract: Despite achieving strong results on standard benchmarks, current point tracking methods rely on feature backbones that are rarely designed with the temporal coherence needed for robust real-world performance. While recent works …

  18. arXiv cs.CV TIER_1 English(EN) · Zengqun Zhao, Ziquan Liu, Yu Cao, Shaogang Gong, Zhensong Zhang, Jifei Song, Jiankang Deng, Ioannis Patras ·

    LatSearch:用于视频扩散模型推理时加速扩展的潜在奖励引导搜索

    arXiv:2603.14526v2 Announce Type: replace Abstract: The recent success of inference-time scaling in large language models has inspired similar explorations in video diffusion. In particular, motivated by the existence of "golden noise" that enhances video quality, prior work has …

  19. arXiv cs.CV TIER_1 English(EN) · Haoran Bai, Xiaoxu Chen, Xiaoyu Liu, Zongsheng Yue, Sibin Deng, Wangmeng Zuo, Ying Chen ·

    SATB-VR:使用信噪比感知轨迹混合训练少步视频恢复扩散模型

    arXiv:2606.28677v1 Announce Type: new Abstract: While diffusion models excel in video restoration, their reliance on extensive iterative steps limits efficiency. Conversely, aggressive single-step distillation often compromises fine texture recovery. To achieve an optimal balance…

  20. arXiv cs.CV TIER_1 English(EN) · Xingang Pan ·

    视频扩散模型在手部动作重建方面的惊人有效性

    4D hand motion reconstruction from egocentric video is bottlenecked by clear limitations of existing methods: image-based pipelines depend on a detector that fails under heavy occlusion, while video-based methods rely on temporal modules learned only from scarce hand-pose annotat…

  21. arXiv cs.CV TIER_1 English(EN) · Xi Ye, Wenjia Yang, Yangyang Xu, Xiaoyang Liu, Duo Su, Mengfei Xia, Jun Zhu ·

    SHIFT:视频扩散模型中的运动对齐与对抗性混合微调

    arXiv:2603.17426v2 Announce Type: replace Abstract: Image-conditioned video diffusion models achieve impressive visual realism but often suffer from weakened motion fidelity, e.g., reduced motion dynamics or degraded long-term temporal coherence, especially after fine-tuning. We …

  22. arXiv cs.CV TIER_1 English(EN) · Ruoyu Wang, Jialun Liu, Huayang Huang, Haibin Huang, Jiepeng Wang, Chi Zhang, Xuelong Li, Yu Wu ·

    SIFT:用于视频扩散模型中物理上可行的运动的自想象微调

    arXiv:2606.27741v1 Announce Type: new Abstract: Recent advances in video diffusion models have greatly improved visual fidelity, yet their generated motions often violate physical plausibility. We observe a common kinematic failure, "motion entanglement", the unintended coupling …

  23. arXiv cs.CV TIER_1 English(EN) · Yu Wu ·

    SIFT:用于视频扩散模型中物理上可信运动的自想象微调

    Recent advances in video diffusion models have greatly improved visual fidelity, yet their generated motions often violate physical plausibility. We observe a common kinematic failure, "motion entanglement", the unintended coupling of independent motion sources, such as camera mo…