A Finetuned SpeechLLM for Joint Multi-Granular L2 Assessment and Natural-Language Rationales
Researchers have developed a SpeechLLM designed for assessing L2 speech proficiency across multiple granularities and providing natural language rationales. This model, trained using a hybrid approach of supervised fine-tuning and Bounded Direct Preference Optimization, can predict sentence-level labels for accuracy, fluency, and prosody, as well as word/phoneme-level accuracy. While the model demonstrates strong performance and plausible sentence-level rationales, its faithfulness degrades at the word/phoneme level due to sparse and weakly aligned references. AI
IMPACT Introduces a novel approach to automated L2 speech assessment with explainability, potentially improving language learning tools.