FLORES-200+
PulseAugur coverage of FLORES-200+ — every cluster mentioning FLORES-200+ across labs, papers, and developer communities, ranked by signal.
1 day(s) with sentiment data
-
African languages face significant tokenization penalty in frontier LLMs
A new research paper reveals a significant "African Language Tax" in frontier large language models, where tokenizers assign substantially more subword tokens to African languages compared to English. This results in hi…
-
New benchmark HardMTBench stress-tests Chinese-English translation
Researchers have introduced HardMTBench, a new benchmark designed to evaluate Chinese-English machine translation systems on knowledge-intensive domains. Existing benchmarks like FLORES-200 show saturation, with most la…
-
New Somali language corpus and tools released for research
Researchers have developed SomaliWeb v1, a new corpus of Somali text containing approximately 303 million tokens. This dataset was created through a reproducible six-stage pipeline, filtering data from HPLT v2, CC100, a…