DyaPlex: Full-Duplex Speech-Motion Model for Dyadic Interaction
Researchers have developed DyaPlex, a novel full-duplex model capable of simultaneously processing and generating speech and physical motion in real-time. This model integrates a foundational speech model with a new motion pathway, utilizing a dual-tower Transformer architecture. Trained on the Seamless Interaction dataset, DyaPlex achieves synchronized multi-modal interaction and sets new benchmarks for dyadic human interaction. AI
IMPACT Introduces a new architecture for synchronized multi-modal AI interaction, potentially advancing human-AI communication.