SILX AI has released Quasar-Preview, the initial public model in its Quasar Foundation Model series. This early checkpoint showcases the Quasar architecture, featuring a sparse Mixture-of-Experts (MoE) design with approximately 18 billion total parameters and 2 billion active parameters. It incorporates a hybrid recurrent and attention layer configuration, including Loop Transformer and Quasar hybrid attention, and an experimental 5 million token context window. AI
IMPACT Demonstrates advancements in MoE and long-context architectures, potentially influencing future model development.
RANK_REASON Model release from a new lab (SILX AI) with specific architectural details and context window size. [lever_c_demoted from frontier_release: ic=1 ai=1.0]
Read on Hugging Face Trending Models →
- Bittensor SN24
- Docker Model Runner
- Google Colab
- Hugging Face
- Kaggle
- OpenAI
- SGLang
- SILX AI
- silx-ai/Quasar-Preview
- transformers
- vLLM
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →