Llama.cpp adds MTP, new Gemma-4 finetune released, Qwen 3.6 excels locally

The llama.cpp project has integrated Multi-head Attention Parallelism (MTP), leading to an 11.5% speed increase for 27B Qwen models in local inference. A new finetuned Gemma-4 model, optimized for creative writing and available in GGUF format, has been released for use with Ollama. Additionally, Qwen 3.6 models have demonstrated competitive performance on the Terminal-Bench 2.0 leaderboard, even surpassing Gemini 2.5 Pro in certain local coding tasks. AI

Summary written by gemini-2.5-flash-lite from 1 source. How we write summaries →

IMPACT Local LLM inference performance is boosted by llama.cpp's MTP integration, while new finetunes and benchmark results highlight community-driven model specialization.

RANK_REASON The cluster details updates to open-source LLM inference software and new finetuned models, along with benchmark results. [lever_c_demoted from research: ic=1 ai=1.0]

Read on dev.to — LLM tag →

• llama.cpp
• Qwen
• Gemma-4
• Ollama
• Gemini 2.5 Pro
• Terminal-Bench 2.0

• model release
• infra