New framework evaluates singing performance using lyrics and music

Researchers have developed MusicJudge, a new framework designed to automatically evaluate singing performance by considering both lyrical correctness and musical fidelity. Unlike previous systems that focused on either acoustic cues or lyric transcriptions, MusicJudge integrates these modalities through block-aligned multimodal analysis. The framework uses multi-signal matching, incorporating semantic embeddings, lexical similarity, and phonetic alignment to identify lyric blocks, and employs Modality-Guided LoRA for fine-tuning Automatic Speech Recognition (ASR) to improve singing audio transcription. Experiments show that MusicJudge aligns well with human expert judgments and demonstrates generalizability across different datasets. AI

IMPACT This framework could advance automated music education and performance analysis tools.

RANK_REASON Academic paper detailing a new framework for singing performance evaluation. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• arXiv
• MusicJudge

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →