New SQuTR benchmark tests spoken query retrieval robustness against noise

Researchers have introduced SQuTR, a new benchmark designed to evaluate the robustness of spoken query to text retrieval systems under various acoustic noise conditions. The benchmark includes a large dataset of over 37,000 queries from existing retrieval datasets, synthesized speech from 200 speakers, and 17 categories of real-world environmental noise. Evaluations using SQuTR revealed that retrieval performance degrades significantly with increasing noise levels, highlighting robustness as a critical bottleneck for current systems, even large-scale models. AI

IMPACT This benchmark will facilitate research into making spoken query systems more reliable in noisy environments.

RANK_REASON The cluster describes a new academic paper introducing a benchmark for evaluating AI systems. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• arXiv
• Ke Yang
• SQuTR

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →