llama.cpp adds GDN chunked kernel, boosting LLM inference speeds up to 20% · 1 source tracked

A pull request for the llama.cpp project introduces a new "GDN chunked kernel" that significantly boosts inference performance across various hardware configurations. Benchmarks show improvements ranging from 3.4% to over 20.6% for models like Qwen3.5-35B and Qwen3.8-27B when using different prompt token lengths and quantization levels. This optimization targets CUDA and Heterogeneous Integration Platform (HIP) architectures, aiming to enhance the efficiency of local large language model deployments. AI

IMPACT Enhances local LLM inference speeds, potentially enabling more complex models on consumer hardware.

RANK_REASON Pull request for an open-source project implementing a performance optimization for LLM inference. [lever_c_demoted from research: ic=1 ai=1.0]

Read on r/LocalLLaMA →

• am17an
• CUDA
• ggml-org
• Heterogeneous Integration Platform
• llama.cpp
• RTX PRO 6000 Blackwell
• Qwen3.5-35B-A3B-Q4_K_S
• Qwen3.5-35B-A3B-Q8_0
• Qwen3.6-35B-A3B-UD-Q4_K_M
• Qwen3.8-27B-Q4_K_M
• Qwen3.8-27B-Q8_0
• Qwen3.8-27B-UD-Q8_K_L
• Qwen3.8-Flash-Next-UD-IQ4_XS
• RTX 4090
• RTX 5090
• Spark GB10

• infra
• product

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →