Local LLM Benchmarks Show MoE Models Struggle with Memory Bandwidth

A user on a Surface Laptop Ultra has benchmarked eight different large language models (LLMs) running locally using llama.cpp. The tests focused on memory bandwidth, revealing that dense models achieved approximately 89% of the system's theoretical memory ceiling, while Mixture-of-Experts (MoE) models only reached 43-67%. This significant performance gap highlights the overhead associated with the routing mechanisms in MoE architectures. AI

IMPACT Highlights the performance cost of MoE architectures in memory-bound scenarios.

RANK_REASON User-generated benchmark analysis of LLM performance on specific hardware. [lever_c_demoted from research: ic=1 ai=0.7]

Read on dev.to — LLM tag →

• Blackwell RTX
• Grace CPU
• llama.cpp
• LPDDR5X-9400
• Nemotron 3.5 Super VL
• Nvidia
• Ollama
• Qwen3.6 35B-A3B
• Surface Laptop Ultra
• Windows 11 Arm64

• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →