Belebele
PulseAugur coverage of Belebele — every cluster mentioning Belebele across labs, papers, and developer communities, ranked by signal.
3 day(s) with sentiment data
-
Qwen3 adaptation to Tajik shows mixed results on comprehension
Researchers explored adapting the Qwen3 language model to the Tajik language, finding that while fine-tuning with LoRA adapters significantly reduced text loss, it did not consistently improve reading comprehension on T…
-
New Tajik language embedder improves retrieval accuracy on legal texts
Researchers have developed Saidzoda-Tajik-E5-Large, a 560 million parameter text embedder specifically fine-tuned for the Tajik language. This model significantly improves retrieval accuracy on Tajik legal and tax docum…
-
LiquidAI d1-Omni-2B: A decision model that hears but doesn't speak
The LiquidAI d1-Omni-2B model, designed as a "decision model" rather than a traditional chat assistant, can process various inputs including audio but does not generate spoken or written responses. It answers a set of p…
-
TypeSafe AI's Jev model outperforms open-source alternatives on decision-making tasks · 4 sources tracked
TypeSafe AI has released Jev, a "System One" model designed for making small, specific decisions rather than generating text. Jev returns choices from a fixed set, scores on a rubric, or calibrated probabilities, aiming…
-
LLM language support claims differ from official commitments
Large language models often claim to support over a hundred languages due to their extensive pretraining data, but this fluency does not equate to official support or verified benchmark performance. Vendors like Meta (L…
-
AI benchmarks cover less than 3% of world languages, study finds
Current AI language model benchmarks significantly underrepresent the world's linguistic diversity, with the broadest benchmarks covering only about 2.9% of the roughly 7,000 living languages. Even the most comprehensiv…
-
Indian languages face 8x "tokenizer tax" in LLMs due to English-centric training
A new research paper highlights a significant disadvantage faced by Indian languages when processed by large language models due to subword tokenization. These tokenizers, primarily trained on English data, result in an…
-
LangMAP tokenization adapts multilingual models without vocabulary changes
Researchers have developed LangMAP, a novel language-adaptive tokenization approach that extends the UnigramLM algorithm for multilingual settings. This method allows for language-specific tokenization from a single sha…
-
Multilingual Code-Switching Boosts LLM Performance Across Four Languages
Researchers have explored the impact of multilingual code-switching data (CSD) on large language models (LLMs) across four languages: English, Japanese, Korean, and Chinese. Their experiments demonstrated that incorpora…
-
New research tackles multilingual adaptation in Mixture-of-Experts models
Two new research papers explore the adaptation of Mixture-of-Experts (MoE) models for multilingual tasks. One paper analyzes how language specialization emerges in MoE models during continual pre-training, finding that …