ggml-org/llama.cpp
PulseAugur coverage of ggml-org/llama.cpp — every cluster mentioning ggml-org/llama.cpp across labs, papers, and developer communities, ranked by signal.
2 day(s) with sentiment data
-
30.5B MoE LLM runs at 39.2 tok/s on 4GB VRAM GPU
A developer has successfully optimized a 30.5 billion parameter Mixture-of-Experts (MoE) Large Language Model (LLM), Qwen3-30B-A3B, to run at 39.2 tokens per second on a standard office workstation with only 4 GB of VRA…
-
llama.cpp CUDA optimizations boost low-bit type performance
A pull request to the llama.cpp project introduces optimizations for CUDA, specifically targeting faster MMVQ (Matrix Multiplication Vector Quantization) for low-bit data types. This enhancement is expected to significa…
-
Llama-server prompt cache bug causes incorrect LoRA scale inference
A bug in the `llama-server`'s RAM prompt cache can lead to incorrect text generation when using LoRA adapters at different scales. The cache stores computed KV states but does not record the specific LoRA adapter and sc…
-
Llama-server bug misreports logprobs with speculative decoding
A bug in the llama-server software causes speculative decoding to incorrectly report log probabilities as zero for generated tokens. This issue affects various models and configurations, including Gemma 3B, when specula…
-
Llama.cpp project releases v0.1.0 amid AI hype and skepticism
A new release of the Llama.cpp project, version 0.1.0, has been announced, touting AI-driven code creation capabilities. The release is framed with significant hype, using terms like "groundbreaking" and "revolutionize,…
-
llama.cpp update optimizes tensor operations with reduced synchronization
A recent update to the llama.cpp project, specifically pull request #20793, introduces significant optimizations for tensor operations. These changes aim to reduce synchronization overhead during split computations, par…
-
llama.cpp integrates Gemma 4 MTP for faster local model performance
The llama.cpp project has merged support for Gemma 4 MTP, a feature that enhances the speed and efficiency of local large language models. This integration allows users to leverage Gemma 4 with Quantization Aware Traini…
-
Granite Vision 4.1 4B model offers efficient document extraction
Granite Vision 4.1 4B is a new vision-language model designed for efficient structured document extraction. It excels at tasks like chart, table, and key-value pair extraction, offering competitive performance in a comp…
-
New 'vintage' LLM trained on pre-1931 English text
A new language model, talkie-1930-13b-it, has been developed, trained exclusively on English text from before 1931. This 13 billion parameter model is an instruction-tuned version of talkie-1930-13b-base, which was init…