Research paper reveals length bias in machine translation quality estimation metrics

A new research paper has identified a systematic bias in quality estimation (QE) metrics used for machine translation. These metrics tend to over-predict errors in longer translations, even when the translations are high-quality. Furthermore, some metrics favor shorter translations while others favor longer ones, irrespective of actual quality. This length bias can negatively impact downstream processes like data filtering and system optimization. The researchers propose a length normalization technique during training as a solution to decouple error prediction from translation length. AI

IMPACT Highlights potential inaccuracies in automated evaluation metrics, impacting the reliability of machine translation systems and data selection pipelines.

RANK_REASON The cluster contains a research paper detailing findings on a specific technical issue within AI/ML. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• arXiv
• Hugging Face
• LLM-as-a-Judge
• MetricX-24
• Quality Estimation
• Yilin Zhang

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →