Researchers have introduced IQuest-Q1, an open-source agentic foundation model designed for command-line interface (CLI) systems. This model utilizes a sparse Mixture-of-Experts (MoE) architecture with 320 billion total parameters, activating 15 billion at a time. Its development involved a three-stage process including stable pre-training with specific normalization techniques and curriculum learning, fine-grained mid-training to expand data mixtures and context length, and multi-scaffold post-training combining supervised fine-tuning and reinforcement learning. The resulting model demonstrates proficiency in both coding and general tasks. AI
IMPACT This open-source release of IQuest-Q1 could accelerate research and development in agentic CLI systems and foundation models.
RANK_REASON The cluster describes the release of a new open-source AI model detailed in an arXiv paper and on Hugging Face.
Read on Hugging Face Daily Papers →
- arXiv
- Hugging Face
- Mixture-of-Experts
- Muon
- reinforcement learning
- Stable SplitNorm
- supervised fine-tuning
AI-generated summary · Google Gemini · from 2 sources. How we write summaries →
