Local LLM speed tied to active parameters, not total size on Windows on Arm

A performance test on a Surface Laptop Ultra revealed that the generation speed of local large language models is more closely tied to their active parameter count per token rather than their total size. The GPT-OSS 120B model, for instance, ran approximately 3.9 times faster than the Qwen3.8-27B model, despite having a significantly larger total parameter count. However, speed did not always correlate with capability, as the fastest model struggled to fix a simple one-character configuration error. AI

IMPACT This benchmark suggests that for local LLM deployments, optimizing for active parameters could yield significant speed improvements without necessarily requiring larger total model sizes.

RANK_REASON The item details a performance benchmark of multiple LLMs on specific hardware, comparing their speed and capability. [lever_c_demoted from research: ic=1 ai=1.0]

Read on dev.to — LLM tag →

• GPT-OSS 120B
• Hermes 4.3 36B
• llama.cpp
• Nemotron 3.5 Super VL
• Nvidia RTX Spark N1X
• Ollama
• Qwen3.6-35B-A3B
• Qwen3.8-27B
• Surface Laptop Ultra
• Windows On Arm

• infra
• product

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →