Researchers have introduced DuplexLedger, a new benchmark designed to evaluate the transactional memory capabilities of full-duplex speech agents. The benchmark comprises 1,094 sessions across four scenarios, providing detailed event-level supervision and memory states. Evaluations of five existing models revealed a significant gap, with no system effectively managing persistent memory while simultaneously listening and speaking, and the best achieving only 2.01% final-state fact coverage. To address this, the team proposed ACT-Mem, an 8.92M-parameter sidecar model that improved latest-value accuracy to 8.68% on held-out sessions, though it introduced some latency. AI
IMPACT Highlights a critical challenge in developing conversational AI: managing persistent memory effectively during real-time interactions.
RANK_REASON The item is a research paper introducing a new benchmark and a proposed model for evaluating AI agents. [lever_c_demoted from research: ic=1 ai=1.0]
- alphaXiv
- arXiv
- CatalyzeX
- Connected Papers
- DagsHub
- DuplexLedger
- Gotit.pub
- Hugging Face
- Litmaps
- MiniCPM-o 4.5
- ScienceCast
- Scite
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →