Llama.cpp adds MTP, new Gemma-4 finetune released, Qwen 3.6 excels locally

The llama.cpp project has integrated Multi-head Attention Parallelism (MTP), leading to an 11.5% speed increase for 27B Qwen models in local inference. A new finetuned Gemma-4 model, optimized for creative writing and available in GGUF format, has been released for use with Ollama. Additionally, Qwen 3.6 models have demonstrated competitive performance on the Terminal-Bench 2.0 leaderboard, even surpassing Gemini 2.5 Pro in certain local coding tasks. AI

IMPACT Local LLM inference performance is boosted by llama.cpp's MTP integration, while new finetunes and benchmark results highlight community-driven model specialization.

RANK_REASON The cluster details updates to open-source LLM inference software and new finetuned models, along with benchmark results. [lever_c_demoted from research: ic=1 ai=1.0]

Read on dev.to — LLM tag →

• Gemini 2.5 Pro
• llama.cpp
• Ollama
• Qwen
• Terminal-Bench 2.0
• Gemma-4

• model release
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →