Speech-to-SFT pipeline ablation reveals data quality gains don't always boost downstream performance

A new research paper published on arXiv details a factorial ablation study of a speech-to-SFT pipeline, investigating the impact of different refinement stages on data quality and downstream model performance. The study found that while improvements in QA data quality consistently increased scores from LLM judges, these gains did not uniformly translate to better performance on downstream multiple-choice QA benchmarks. The positive transfer of improvements was concentrated on family-domain aligned pairs, suggesting a potential format mismatch between the SFT data composition and the nature of MCQA probes. AI

IMPACT Investigates how different stages of speech-to-SFT data refinement impact downstream model performance, highlighting potential mismatches between data quality improvements and benchmark gains.

RANK_REASON Research paper detailing a factorial ablation study of a speech-to-SFT pipeline. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• A Factorial Ablation of a Speech-to-SFT Pipeline: Differential Effects on Data Quality and Downstream Transfer
• arXiv
• Hugging Face
• Korean
• MCqasim
• supervised fine-tuning
• Whisper-medium

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →