New PitchBench Benchmark Reveals Unreliable Pitch Hearing in Audio-Language Models

Researchers have developed PitchBench, a new evaluation suite designed to systematically measure the pitch perception abilities of audio-language models (ALMs). The suite includes 28 experiments that test both absolute and relative pitch identification across various conditions, such as different instruments, noise levels, and musical textures. Initial evaluations using PitchBench revealed that current ALMs exhibit unreliable pitch hearing, performing poorly and inconsistently across tasks, indicating that stable pitch perception is not yet achieved in these models. AI

IMPACT Highlights a critical gap in current audio-language models, potentially guiding future research towards more robust auditory perception capabilities.

RANK_REASON The cluster contains a research paper introducing a new benchmark for evaluating AI models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• arXiv
• Audio-language models
• PitchBench

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →