llama.cpp releases updates with performance and stability fixes · 9 sources tracked
ByPulseAugur Editorial·[125 sources]·
The llama.cpp project has released several updates, including version 0.4.1, which addresses various performance and stability issues across different platforms. Notable changes include optimizations for SYCL backends, fixes for heap corruption on CPU, and fallback mechanisms for CUDA devices lacking BF16 hardware acceleration. These updates aim to improve the efficiency and reliability of running language models on diverse hardware.
AI
IMPACT
These updates enhance the performance and stability of llama.cpp, potentially improving the efficiency of running local LLMs on diverse hardware.
RANK_REASON
The cluster consists of multiple release notes for the llama.cpp project, detailing bug fixes and minor improvements rather than a new model release or significant research breakthrough.
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Tool
The cluster consists of multiple release notes for the llama.cpp project, detailing bug fixes and minor improvements rather than a new model release or significant research breakthrough.
Source corroboration
125 independent sources
Strong cross-source corroboration — multiple independent publishers covered this within the clustering window.
Topics
product, infra
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
50 days old
Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.
Coverage growth since scoring
+22 source(s) since last score
New sources have picked up this story since our last re-score. Score will update on the next scoring pass.
<details open=""> <p>HIP: broaden MoE ncols_opt tile heuristic on RDNA3.5 architecture (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/28935">#28935</a>)</p> <p>It's found the MoE ncols_opt tile heuristic needs to be broadened<br /> to includ…
<details open=""> <p>vulkan: make MUL_MAT_ID BN/2 tail unconditional (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/28923">#28923</a>)</p> <p>Use BN/2 as the default for BNover2 and as the disabled fallback for BNover4, and remove the enable…
<details open=""> <p>metal: fix NaN in mul_mm_id when activations exceed f16 range (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/26223">#26223</a>)</p> <ul> <li>test-backend-ops: reproduce MUL_MAT_ID NaN for activations beyond f16</li> </ul…
<details open=""> <p>hexagon: add back missing contiguous fast-path and hvx_copy_uu for each run (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/28886">#28886</a>)</p> </details> <p><strong>Website:</strong></p> <ul> <li><a href="https://llam…
<details open=""> <p>models : move build_arch_graph() after graph() template specialization (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/28934">#28934</a>)</p> <p>Move build_arch_graph()'s function definitions after the graph<br /> and gra…
<details open=""> <p>sycl: rfc: Use radix select for top_k (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/28670">#28670</a>)</p> <ul> <li>sycl: GPU-resident TOP_K for large k, parallelised over the device</li> </ul> <p>The SYCL backend refus…
<details open=""> <p>common : move llama_n_rs_seq to before llama_decode (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/28749">#28749</a>)</p> <p>This commit moves the llama_n_rs_seq function call to before the<br /> llama_decode call and re…
<details open=""> <p>ggml-cuda: fallback to F32 on device without BF16 hardware acceleration (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/28846">#28846</a>)</p> <ul> <li> <p>ggml-cuda: fallback to F32 on device without BF16 hardware accele…
<details open=""> <p>models : guard the expert FFN size fallback in nemotron-h against a zero divisor (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/28779">#28779</a>)</p> <p>The NextN/MTP tail loop derives the expert FFN size as n_ff/n_expe…
<details open=""> <p>vulkan: workaround NV queuesubmit driver bug (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/28830">#28830</a>)</p> <p>There is a driver bug where two queues on the same VkDevice simultaneously<br /> submitting can break …
<details open=""> <p>opencl: apply the noshuffle row-alignment rule to q4_K, q5_K and q8_0, not just q6_K (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/28575">#28575</a>)</p> </details> <p><strong>Website:</strong></p> <ul> <li><a href="htt…
<details open=""> <p>cmake: leave the timestamp out of precompiled headers on clang (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/28816">#28816</a>)</p> <p>Clang stores the modification time of the precompiled header sources<br /> inside th…
<details open=""> <p>server: frame the router child state command as a whole line (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/28747">#28747</a>)</p> <p>The child writes its state commands on stdout while the logger writes<br /> on stderr,…
<details open=""> <p>webgpu: align tensor bindings to the type block size (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/28382">#28382</a>)</p> <p>Walk the binding offset back until the distance to the tensor is a<br /> whole number of block…
<details open=""> <p>ggml-webgpu: Update to a recent version of Dawn (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/28683">#28683</a>)</p> <ul> <li> <p>ggml-webgpu: Update to a recent version of Dawn</p> </li> <li> <p>No module scanning</p> …
<details open=""> <p>cmake : skip PCH for llama-server PCH when using MSVC (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/28763">#28763</a>)</p> <p>This commit fixes an issue that I introduced when adding PCH<br /> (precompiled headers) in C…
<details open=""> <p>metal : fix idle threads in the remaining iq mul_mv kernels for ne00 < 1024 (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/28692">#28692</a>)</p> <ul> <li>metal : fix idle threads in the remaining iq mul_mv kernels fo…
<details open=""> <p>server: fix speculation after an image (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/28715">#28715</a>)</p> <ul> <li>server: fix speculation after an image</li> </ul> <p>Pass the actual position to the drafter after an …
<details open=""> <p>vulkan: fix data race and OOB access in argsort(large) (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/28705">#28705</a>)</p> <p>argsort had a data race in the inner loop, which VVL caught. But I don't think<br /> this wa…
<details open=""> <p>vulkan: use CPU writes in ggml_backend_vk_cpy_tensor_async if the context is idle (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/28618">#28618</a>)</p> </details> <p><strong>Website:</strong></p> <ul> <li><a href="https:…
<details open=""> <p>vulkan: small M matrix optimizations for qwen (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/28457">#28457</a>)</p> <ul> <li> <p>vulkan: optimize m=1 mul_mat by swapping A/B</p> </li> <li> <p>vulkan: Improve small M perf…
<details open=""> <p>ci : Update WoA CUDA 13.4 release to use 13.4.1 GA redistributables (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/28687">#28687</a>)</p> <ul> <li>Move Windows ARM64 CUDA 13.4 builds from the Developer Preview archives t…
<details open=""> <p>models: clean up some dead switch branches in old models (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/28669">#28669</a>)</p> <p>Some of these if statements were copypastaed in a former refactor and<br /> never cleaned …
<details open=""> <p>vulkan: use spec constant for matrix matrix multiplication A-type (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/25773">#25773</a>)</p> <ul> <li>vulkan: use spec constant for mul mat type_a</li> </ul> <p>vulkan: use map …
<details open=""> <p>vulkan: Convert FILL to distribute workgroups in 2D to avoid exceeding maxComputeWorkGroupCount (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/28592">#28592</a>)</p> <ul> <li>divide workload to 2D</li> </ul> <p>This is t…
<details open=""> <p>llama : use int32_t for llama_sampler_chain_n return type (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/28631">#28631</a>)</p> <p>Contributes to <a class="issue-link js-issue-link" href="https://github.com/ggml-org/llam…
<details open=""> <p>CUDA: replace GGML_FA_ALL_QUANTS with GGML_FA_QUANTS, more control over what is compiled (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/28079">#28079</a>)</p> <ul> <li>CUDA: add configurable FA quant combinations</li> </…
<details open=""> <p>jinja: treat a null left operand of in as a plain lookup (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/28620">#28620</a>)</p> <p>Templates that default an optional variable to none and then test its<br /> membership in …
<details open=""> <p>tests : use less threads for data initialization (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/28325">#28325</a>)</p> <ul> <li> <p>tests : use 1 thread for data initialization</p> </li> <li> <p>cont : scale threads with…
<details open=""> <p>server : apply checkpoint min-step eviction only when the checkpoint list is full (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/28302">#28302</a>)</p> <p>The spacing eviction in create_checkpoint() keeps the oldest chec…
<details open=""> <p>ggml: add gfx90c HIP support (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/26454">#26454</a>)</p> <ul> <li> <p>ggml: add gfx90c HIP support</p> </li> <li> <p>ggml: make gfx90c HIP support compliant with specifications</…
<details open=""> <p>CUDA: branchless Q4_K/Q5_K unpack to speed up mmvq, L2 prefetch on DGX Spark (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/26705">#26705</a>)</p> <ul> <li> <p>Update Q4_K and Q5_K to use branchless computation, which st…
<details open=""> <p>vulkan: support type-aligned GET_ROWS (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/28253">#28253</a>)</p> <ul> <li>vulkan: fall back to CPU for GET_ROWS with misaligned offsets</li> </ul> <p>The Vulkan GET_ROWS shader …
<details open=""> <p>models : fix GDN normalization from <code>max</code> to <code>rsqrt</code> (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/28068">#28068</a>)</p> <ul> <li>models: use flash-linear-attention's l2norm for gated delta net q/…
<details open=""> <p>sycl: attribute device allocations by site (GGML_SYCL_MEMTRACE) (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/27631">#27631</a>)</p> <p>define two new environment variables to better understand how much<br /> memory is …