Bilibili has released Index-1.9B, a series of four open small language models, including a base model, a control variant, a chat-aligned model, and a character-focused model. The base model, trained on 2.8 trillion tokens, achieves a competitive average benchmark score of 64.92 despite its 1.9 billion parameters. The technical report details training methodologies, including a novel learning-rate schedule and output layer, and presents controlled studies on various training parameters. All models and evaluation code are publicly available. AI
IMPACT Provides new open-source models for researchers and developers, potentially advancing capabilities in Chinese and English language tasks.
RANK_REASON The cluster contains a technical report detailing the release of new open small language models. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
