llama.cpp
PulseAugur coverage of llama.cpp — every cluster mentioning llama.cpp across labs, papers, and developer communities, ranked by signal.
- 2026-06-08 research_milestone llama.cpp merged a pull request to optimize KV cache performance for the Gemma-4 model. source
- 2026-06-05 product_launch A SYCL backend has been ported to llama.cpp, offering performance improvements for Intel Arc GPUs. source
- 2026-05-30 product_launch llama.cpp released version b9438, adding custom CSS injection for web UI theming. source
- 2026-05-25 research_milestone A fix is expected for llama.cpp to address split mode tensor crashes. source
- 2026-05-25 product_launch A pull request was submitted to improve checkpoint creation and context handling in llama.cpp. source
- 2026-05-24 product_launch llama.cpp released version b9305 with pre-compiled binaries for multiple platforms. source
- 2026-05-17 research_milestone llama.cpp implements MTP optimizations and prompt decode improvements for faster local AI inference. source
- 2026-05-14 product_launch A performance-optimized fork of llama.cpp was released with new features. source
- 2026-05-12 product_launch llama.cpp project integrates llama-eval tool for model benchmarking. source
31 day(s) with sentiment data
-
llama.cpp server enables sub-30-second model hot-swapping
The llama.cpp server now supports hot-swapping models in under 30 seconds, a significant improvement over previous methods. This feature allows for rapid model changes without needing to restart the server. The update i…
-
Developer implements KVarN KV-cache compression in llama.cpp fork
A developer has implemented Huawei's KVarN KV-cache quantization technique in a fork of the llama.cpp project, named BeeLlama.cpp. This implementation allows users to compress KV caches by 3-5 times, aiming to reduce VR…
-
Unsloth releases optimized Gemma 4-31B model with integration guides
Unsloth has released a quantized version of the Gemma 4-31B model, optimized for efficient inference. This release provides detailed instructions and code examples for integrating the model into various popular AI libra…
-
OBLITERATUS/Gemma-4-12B-OBLITERATED model available on Hugging Face
The OBLITERATUS/Gemma-4-12B-OBLITERATED model is now available on Hugging Face, offering users detailed instructions for integration with various libraries and tools. These include popular frameworks like Transformers a…
-
Gemma 4 12B praised for ease of use in local coding
A user on the r/LocalLLaMA subreddit has found Gemma 4 12B to be their preferred model for local coding tasks, surpassing previous models like Qwen 3.6 27B. The user highlights Gemma 4's ease of use, particularly its pl…
-
Quantizing spec draft may reduce MTP context size, user finds
A user on the r/LocalLLaMA subreddit discovered that quantizing the spec draft when using MTP (likely a model inference framework) can unexpectedly reduce context size. The user found that disabling this quantization in…
-
New tool optimizes llama.cpp models with advanced NVFP4/MXFP6 quantization
A developer has released an advanced quantizer tool for llama.cpp, designed to create NVFP4 and MXFP6 GGUF models. This tool goes beyond basic quantization by evaluating various methods and incorporating custom techniqu…
-
BeeLlama v0.3.1 boosts local LLM performance with DFlash, MTP
BeeLlama v0.3.1, a fork of llama.cpp, has been released with significant performance enhancements. This update integrates features like DFlash, Multi-Threaded Processing (MTP), and new quantization options such as q6_0 …
-
Qwen 3.6 35B model excels with KV cache in agentic tasks
A user on r/LocalLLaMA found that the Qwen 3.6 35B model significantly outperforms the 27B version, particularly in agentic tasks, when using KV cache. This user initially favored the 27B model for its perceived intelli…
-
llama.cpp gains 28% context with OpenBLAS build
A user on Reddit's r/LocalLLaMA subreddit has discovered that compiling the llama.cpp software with OpenBLAS support, in addition to Vulkan, allows for a significant increase in context window size. When using the Qwen …
-
User doubles LLM inference speed by fixing PCIe slot bottleneck
A user building a multi-GPU setup for local LLM inference discovered a significant performance bottleneck caused by a misconfigured PCIe slot. One of the four RTX 3090 GPUs was incorrectly placed in a slot that only sup…
-
Llama-cpp update skips Gemma model reasoning phase
A user on r/LocalLLaMA encountered an issue where the reasoning phase of the Gemma4 31b model was being skipped in recent builds of llama-cpp. This functionality had previously worked, but a recent update related to the…
-
BC250 device performance benchmarked with custom Llama-cpp setup
A user on Reddit shared performance metrics for a BC250 device running Fedora 44 with a customized Llama-cpp setup. The user detailed their process of overclocking the device to 2Ghz and unlocking 40 Compute Units, whic…
-
Jetson AGX Orin 64GB sees faster LLM prefill with q8_0 quantization
A user on the r/LocalLLaMA subreddit shared performance observations for the Jetson AGX Orin 64GB, noting that the q8_0 quantization method for models resulted in significantly faster prompt processing compared to q6_k …
-
Gemma 4 12B local AI model requires configuration tweaks for optimal performance
Google's Gemma 4 12B model shows promise for local AI setups, but users report that default configurations in tools like LM Studio can hinder its reasoning capabilities. Specific adjustments to Jinja templates and sampl…
-
llama.cpp releases b9500 with broad OS and hardware support
The llama.cpp project has released version b9500, offering pre-compiled binaries for a wide range of operating systems and hardware architectures. This release includes support for macOS (Apple Silicon and Intel), Linux…
-
Ideogram 4 safety filters bypassed with local LLM integration
A user found that Ideogram 4's safety filters are not overly restrictive when integrated with a local LLM like Gemma-4-31B. By bypassing the default LLM and using a custom API call with minor modifications to Ideogram's…
-
llama.cpp b9501 refactors state saving tests for token input
The llama.cpp project has released version b9501, which includes refactoring for its test-save-load-state functionality. This update allows the test to accept token input, defaulting to generating random tokens if no pr…
-
llama.cpp user questions parallel setting impact on agent harnesses
A user on the r/LocalLLaMA subreddit is inquiring about the impact of setting the `--parallel` parameter to 1 in llama.cpp. This setting reportedly limits the model to a single user chat at a time but increases context …
-
Local AI models run on consumer GPUs, cutting costs
New advancements in local AI are making large language models accessible on personal hardware. Models like OpenAI's GPT-OSS-120B and Google's Gemma 4 12B are now runnable on consumer-grade GPUs such as the RTX 5090 and …