研究人员开发了一个多模态语音活动预测（MM-VAP）框架，以改进社交机器人的轮次转换预测。该系统通过整合同步的视听输入并利用为该任务改编的预训练语音相关骨干网络，扩展了以前仅限音频的方法。MM-VAP框架采用说话人间注意力来模拟对话动态，并使用语义一致性损失来优化未来的语音活动预测，在NoXi和Haru EDR等数据集上表现出改进的性能。 AI
影响 该框架有望在社交和调解环境中实现更自然、更有效的人机交互。
排序理由 该集群包含一篇详细介绍AI应用新框架的研究论文。
AI 生成摘要 · Google Gemini · 来自 3 个来源。 我们如何撰写摘要 →
arXiv:2607.07294v1 Announce Type: cross Abstract: Turn-taking prediction is a key requirement for social robots involved in human-human interaction, particularly in mediator settings, where the robot must anticipate conversational dynamics rather than merely react to pauses. This…
Turn-taking prediction is a key requirement for social robots involved in human-human interaction, particularly in mediator settings, where the robot must anticipate conversational dynamics rather than merely react to pauses. This work presents a Multimodal Voice Activity Project…
Turn-taking prediction is a key requirement for social robots involved in human-human interaction, particularly in mediator settings, where the robot must anticipate conversational dynamics rather than merely react to pauses. This work presents a Multimodal Voice Activity Project…