Speech Language Models Fail to Grasp Sound Symbolism Like Humans

A new research paper explores whether speech language models (SLMs) can understand sound symbolism, the human ability to associate speech sounds with perceptual qualities like sharpness or roundness. The study found that current SLMs, including open-weight models, do not accurately replicate human judgments based on actual speech recordings. The models failed to identify the acoustic cues, such as spectral tilt, that drive human intuitions, suggesting that improvements in speech representation are needed for better perceptual alignment. AI

IMPACT Highlights limitations in current SLMs' understanding of auditory perception, suggesting a need for improved speech representations.

RANK_REASON Research paper published on arXiv detailing findings about SLMs and sound symbolism. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• arXiv
• Hugging Face
• Small Language Models
• Speech Language Models

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →