Tencent and VIDRAFT showcase sparse MoE models with reduced active parameters

Tencent has released Hy3, a 295-billion-parameter Mixture-of-Experts (MoE) model that utilizes only 21 billion active parameters per forward pass, significantly reducing inference costs. This MoE architecture, featuring 80 layers with 192 routed experts per layer, allows for selective computation, making its operational cost comparable to a 21B dense model. Additionally, Hy3 incorporates a Multi-Token Prediction layer that speeds up token generation and is compatible with existing serving frameworks like vLLM and SGLang, while also offering different reasoning modes and tool-calling capabilities. In parallel, VIDRAFT has demonstrated Ourbox-35B-JGOS, a 34.7B parameter MoE model that can run on a CPU due to its sparse nature, where only approximately 3 billion parameters are active per token. This efficiency is attributed to memory bandwidth optimization, streaming only active experts rather than all weights, which drastically reduces memory traffic compared to dense models. Ourbox-35B-JGOS achieves a competitive score of 86.4% on GPQA Diamond and its performance can be reproduced using public GGUF weights and demos. AI

影响 稀疏MoE架构能够实现更高效的模型部署，可能降低硬件要求并提高先进AI功能的可访问性。

排序理由 该集群描述了新的模型架构及其性能特征，属于研究类别。

在 dev.to — LLM tag 阅读 →

• GGUF
• GPQA Diamond
• Nvidia B200
• Ourbox-35B-JGOS
• Q3_K_M
• VIDRAFT
• VKUE
• Apache Software License 2.0
• H200
• mixture of experts
• Multi Token Prediction
• SGLang
• Tencent
• vLLM

• 模型发布
• 基础设施

AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →