A new research paper explores the optimal linguistic units for encoding speech content, comparing phones, syllables, and words. Using a large corpus of English audiobooks and both n-gram and transformer language models, the study found that words generally achieve the lowest bitrates. However, the research indicates that units larger than syllables but smaller than extensive multi-word phrases offer the most efficient representation when language model capacity is constrained. AI
IMPACT This research could inform the design of more efficient speech and language processing systems.
RANK_REASON The item is an academic paper on arXiv discussing linguistic units and information rates. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →