PulseAugur
EN
LIVE 10:40:26

DyaPlex model synchronizes speech and motion for dyadic interaction

By PulseAugur Editorial · [2 sources] ·

Researchers have developed DyaPlex, a novel full-duplex model capable of simultaneously processing and generating speech and physical motion in real-time. This model integrates a foundational speech model with a new motion pathway, utilizing a dual-tower Transformer architecture. Trained on the Seamless Interaction dataset, DyaPlex achieves synchronized multi-modal interaction and sets new benchmarks for dyadic human interaction. AI

IMPACT Introduces a new architecture for synchronized multi-modal AI interaction, potentially advancing human-AI communication.

RANK_REASON The cluster contains an academic paper detailing a new model.

Read on arXiv cs.CV →

AI-generated summary · Google Gemini · from 2 sources. How we write summaries →

COVERAGE [2]

  1. arXiv cs.CV TIER_1 English(EN) · Koki Nagano, Hongyu Liu, Seonwook Park, Tianye Li, Amrita Mazumdar, Christian Jacobsen, Shengze Wang, Michael Stengel, Rajarshi Roy, Ka Chun Cheung, Simon See, Shalini De Mello ·

    DyaPlex: Full-Duplex Speech-Motion Model for Dyadic Interaction

    arXiv:2606.03874v1 Announce Type: new Abstract: We present DyaPlex, a streaming, full-duplex speech-and-motion model designed for dyadic interaction. To capture the continuous and reciprocal nature of human communication, this full-duplex capability empowers the agent to simultan…

  2. arXiv cs.CV TIER_1 English(EN) · Shalini De Mello ·

    DyaPlex: Full-Duplex Speech-Motion Model for Dyadic Interaction

    We present DyaPlex, a streaming, full-duplex speech-and-motion model designed for dyadic interaction. To capture the continuous and reciprocal nature of human communication, this full-duplex capability empowers the agent to simultaneously perceive and generate both speech and phy…

RELATED TOPICS