Aero Realtime: Duplex multimodal model achieves low-latency streaming

Researchers have developed Aero Realtime, a 4-billion parameter multimodal model designed for low-latency, continuous interaction. Unlike previous models that operate in a turn-based manner, Aero Realtime features a duplex architecture that aligns video, audio, and text on a shared temporal grid. This allows new observations to enter the generation stream in real-time, enabling a more natural and responsive user experience. The model achieves processing lags within 200ms of the source timeline, demonstrating the potential for truly interactive multimodal AI. AI

IMPACT Enables more natural, real-time multimodal interactions by allowing continuous input and output streams.

RANK_REASON Research paper detailing a new model architecture and its performance. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• Aero Realtime
• arXiv
• NVIDIA A6000

• model release
• paper
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →