llama.cpp now loads MTP tensors by default, increasing resource usage

The popular open-source inference engine llama.cpp has updated its default behavior to automatically load Multi Token Prediction (MTP) tensors. This change affects models that have MTP tensors baked in, such as GLM-5.2, hy_v3, qwen35moe, and step35. Previously, these tensors were only loaded if speculative decoding was explicitly enabled. Now, they will be loaded by default, potentially increasing VRAM and RAM usage for users even if they do not utilize MTP functionality. AI

IMPACT Users of llama.cpp with MTP-enabled models may see increased VRAM/RAM usage due to the default loading of MTP tensors.

RANK_REASON This is a change in default behavior for an open-source inference engine, affecting resource usage for specific model types.

Read on r/LocalLLaMA →

• GLM-5.2
• hy_v3
• llama.cpp
• Multi Token Prediction
• qwen35moe
• step35

• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →