研究人员开发了 Motion-Omni，一个将语音生成与全身化身动作相结合的端到端框架。与级联系统不同，Motion-Omni 允许语音和动作生成之间的联合优化，从而实现更协调、更自然的化身动作。该系统以 Qwen2.5-7B-Instruct 模型实例化，在保持低词错误率的同时，展示了更快的响应时间和在动作指标上的竞争力。 AI
影响 通过统一语音和动作生成，实现更自然、响应更快的化身交互。
排序理由 详细介绍用于联合语音和动作生成新框架的学术论文。 [lever_c_demoted from research: ic=1 ai=1.0]
在 Hugging Face Daily Papers 阅读 →
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
完整方法见我们的编辑标准。
Motion-Omni is an end-to-end framework that jointly generates spoken dialogue and full-body co-speech motion from shared hidden states, using scalable pseudo-labeling and a unified evaluation protocol to achieve real-time, aligned responses.
arXiv:2609.04250v1 Announce Type: cross Abstract: An avatar that holds a conversation should decide what to say and to move while saying it, yet these abilities live in separate model families: spoken dialogue models produce speech without motion, and co-speech motion models prod…