Researchers have introduced Arkios, a 1.04 billion parameter language model trained on 150 billion tokens of English and Nepali text. The model utilizes a custom training stack and a Devanagari-aware tokenizer. Evaluations show Arkios outperforms comparable open models on ARC-Easy and ARC-Challenge benchmarks, though this may be due to data overlap with the benchmarks rather than general capability. The study also highlights challenges in evaluating low-resource language models, as standard multiple-choice prompts can yield chance-level results for Nepali comprehension. AI
IMPACT Introduces a new bilingual model and highlights evaluation challenges for low-resource languages.
RANK_REASON The cluster describes a new research paper detailing the creation and evaluation of a novel language model. [lever_c_demoted from research: ic=1 ai=1.0]
- Apache Software License 2.0
- ARC-Challenge
- ARC-Easy
- Devanagari
- English
- Nepali
- OLMo 1B
- Pythia-1.4B
- TinyLlama-1.1B
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →