New cESA protocol streamlines machine translation evaluation

Researchers have developed a new protocol called Contrastive Error Span Annotation (cESA) to improve the human evaluation of machine translation. This method presents annotators with multiple translations of the same source input, allowing them to identify and mark error spans and assign an absolute quality score. A large-scale evaluation of English-to-Japanese translations from 12 models demonstrated that cESA reduces annotation time and noise compared to traditional single-output evaluations, yielding consistent and interpretable model rankings. AI

IMPACT This new evaluation method could lead to more accurate and efficient development of machine translation systems.

RANK_REASON The cluster contains a research paper detailing a new protocol for evaluating machine translation. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• 12 models
• arXiv
• Contrastive Error Span Annotation
• English->Japanese
• Hugging Face
• machine translation

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →