llama.cpp project releases pre-release updates with performance optimizations
ByPulseAugur Editorial·[71 sources]·
The llama.cpp project has released several pre-release updates, including improvements to GPU cache management for MoE experts, optimizations for various quantization types on Apple Silicon, and fixes for Metal fusion operations. These updates also address sampling logic for temperature-zero chains and introduce new vocabulary tokens. The releases cover a wide range of platforms including macOS, iOS, Linux, Android, and Windows, with specific optimizations for Apple Silicon and openEuler.
AI
IMPACT
These updates enhance the performance and compatibility of local LLM inference on various hardware and operating systems.
RANK_REASON
The cluster consists of multiple pre-release updates to the llama.cpp project, focusing on software improvements and optimizations rather than a new model release or significant industry event.
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Tool
The cluster consists of multiple pre-release updates to the llama.cpp project, focusing on software improvements and optimizations rather than a new model release or significant industry event.
Source corroboration
71 independent sources
Strong cross-source corroboration — multiple independent publishers covered this within the clustering window.
Topics
infra, product
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
48 days old
Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.
Coverage growth since scoring
+18 source(s) since last score
New sources have picked up this story since our last re-score. Score will update on the next scoring pass.
<details open=""> <p>cuda : Use byte strides for roll to allow non-contiguous ROLL operations (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/29547">#29547</a>)</p> </details> <p><strong>Website:</strong></p> <ul> <li><a href="https://llama.a…
<details open=""> <p>model : support classifier_activation for rerankers (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/29692">#29692</a>)</p> <ul> <li>model : support classifier_activation for rerankers</li> </ul> <p>Assisted-by: Claude Opu…
<details open=""> <p>ggml-cuda: assign four GDN state columns per warp (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/30087">#30087</a>)</p> <ul> <li> <p>ggml-cuda: assign two GDN state columns per warp</p> </li> <li> <p>ggml-cuda: use 4 GDN…
<details open=""> <p>sycl: remove duplicate block-size defines from op headers (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/29507">#29507</a>)</p> <p>All were duplicates of the values in ggml/src/ggml-sycl/presets.hpp</p> </details> <p><st…
<details open=""> <p>hexagon: enable alloc_buffer_n (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/30126">#30126</a>)</p> <ul> <li> <p>hex-bufs: add support for alloc_buffer_n</p> </li> <li> <p>hex-bufs: add support for splitting large tenso…
<details open=""> <p>chat : name tool and argument parser rules by index (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/30088">#30088</a>)</p> <ul> <li> <p>bugfix: infinite recursion caused by a tool named 'call'(<a class="issue-link js-issu…
<details open=""> <p>cuda: FWHT kernels for block widths above 512 (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/29100">#29100</a>)</p> <p>The CUDA FWHT covers widths 64 to 512. It runs one row per warp and keeps N/32<br /> values per lane,…
<details open=""> <p>metal : few-row MMA mat-mul for the remaining src0 types (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/30065">#30065</a>)</p> <p>The generic few-row MMA kernel works for any type with a 16-weight<br /> dequantizer, so i…
<details open=""> <p>metal : fix MUL_MAT+ADD fusion when the residual is itself a MUL_MAT (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/30100">#30100</a>)</p> <ul> <li>metal : fix MUL_MAT+ADD fusion when the residual is itself a MUL_MAT</li…
<details open=""> <p>[SYCL] fix the issue in mixed different model GPUs in FA (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/29071">#29071</a>)</p> <ul> <li> <p>fix mixed different model GPUs issue</p> </li> <li> <p>Update ggml/src/ggml-sycl…
<details open=""> <p>ggml-webgpu: no dawn native features on wasi (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/27069">#27069</a>)</p> </details> <p><strong>Website:</strong></p> <ul> <li><a href="https://llama.app" rel="nofollow">https://l…
<details open=""> <p>cuda : add BF16 support for XIELU (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/29955">#29955</a>)</p> <p>The XIELU CUDA kernel template is already generic over the element<br /> type; only the F32/F16 type assertion an…
<details open=""> <p>llama : re-reserve the sched when the nextn extraction flags change (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/30020">#30020</a>)</p> <p>The speculative MTP init enables NextN extraction on the target and draft<br />…
<p>HIP: use -O0 for host code in debug builds (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/29795">#29795</a>)</p>
<details open=""> <p>CUDA: Optimize accumulation in mmq for NVFP4 type (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/29857">#29857</a>)</p> <ul> <li> <p>ggml_cuda: optimize accumulation in mmq_vec_dot_fp4_fp4_mma for better performance</p> …
<p>ci : winget urls must be separate strings (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/29978">#29978</a>)</p>
<details open=""> <p>cuda: tile the lightning indexer over keys and tokens for 4 heads (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/29901">#29901</a>)</p> <ul> <li>cuda: tile the lightning indexer over keys and tokens for 4 heads</li> </ul…
<p>webgpu: add MMVQ support for Q1_0/Q5_0/Q5_1/Q3_K/Q5_K/Q6_K/MXFP4 (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/issues/29">#29</a>…</p>
<details open=""> <p>CUDA: use MMVF for thin f16/bf16 mul_mat at small batch size (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/29633">#29633</a>)</p> <ul> <li>CUDA: use MMVF for thin f16/bf16 mul_mat at small batch size</li> </ul> <p>Signe…
<details open=""> <p>llama: support both embd + raw tokens in batch (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/29622">#29622</a>)</p> <ul> <li> <p>llama: support both embd + raw tokens in batch</p> </li> <li> <p>add to test-llama-archs</…