New benchmark MORE evaluates multilingual document parsing across 149 languages

Researchers have introduced MORE, a new benchmark designed to evaluate multilingual document parsing capabilities across 149 languages. This benchmark addresses the current lack of evaluation for models on languages beyond English and Standard Chinese, which is crucial for unlocking global knowledge. MORE is notable for its extensive language coverage, its evaluation of structural elements like tables and code blocks in addition to text, and its use of real-world documents annotated through a human-refined pipeline. Initial evaluations using MORE have established new performance baselines for less common languages and demonstrated the benchmark's utility in identifying model weaknesses. AI

IMPACT Enables more accurate evaluation of vision-language models on a wider range of languages, potentially improving global knowledge accessibility.

RANK_REASON The item describes a new benchmark and dataset for evaluating multilingual document parsing, which falls under academic research. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• alphaXiv
• arXiv
• CatalyzeX Code Finder for Papers
• CORE Recommender
• DagsHub
• English
• Gotit.pub
• Hugging Face
• Influence Flower
• MORE
• ScienceCast
• Standard Chinese
• vision-language model

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →