llama.cpp Metal backend fuses MoE routing for DeepSeek and GLM models

The llama.cpp project has released an update that enhances its Metal backend by fusing various Mixture-of-Experts (MoE) routing mechanisms. This update specifically targets models like DeepSeek-V4, DeepSeek-V4.1, and others such as GLM, DeepSeek-V2, and DeepSeek-V3.2, improving their performance on Apple Silicon hardware. The optimizations include fusing sigmoid and sqrt(softplus) MoE routing with selection bias, and normalizing routing weights with scale-bias, leading to more efficient model execution. AI

IMPACT Optimizes performance of various MoE models on Apple Silicon, potentially improving inference speed and efficiency for users of llama.cpp.

RANK_REASON This is a software update for an open-source project that optimizes existing models on specific hardware, rather than a new model release or significant research breakthrough.

Read on llama.cpp — Releases →

• DeepSeek-V2
• DeepSeek-V3.2
• DeepSeek-V4
• llama.cpp
• M5 Ultra
• Mac Studio
• Minimax
• Muse Spark

• infra
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →