ggml-org
PulseAugur coverage of ggml-org — every cluster mentioning ggml-org across labs, papers, and developer communities, ranked by signal.
9 day(s) with sentiment data
-
llama.cpp PR boosts IQ model prompt processing with AVX2 optimizations · 1 source tracked
A pull request for the llama.cpp project introduces AVX2 optimizations to significantly accelerate prompt processing for IQ models, particularly at large batch sizes. Benchmarks show dramatic speed increases, with some …
-
llama.cpp adds CPU offload option for dense models
A new pull request has been submitted to the llama.cpp project, proposing the addition of a `--n-cpu-ffn` option. This feature aims to provide CPU offloading capabilities for dense models, similar to the existing `--n-c…
-
Open-source projects dotenvy and llama.cpp see significant updates
The open-source project dotenvy is being forked into a new project called dotenv-ng, with the developers citing a desire for a fresh start and improved maintenance. Separately, the llama.cpp project has released version…
-
llama.cpp CI job fixed for dry-run reporting
This commit addresses an issue in the make-release continuous integration job within the llama.cpp project. Specifically, it corrects how the job reports its status when the --dry-run flag is activated. Previously, the …
-
Kimi-K3 text model integrated into llama.cpp via pull request
A pull request has been submitted to the llama.cpp project to integrate the Kimi-K3 text model. This contribution, made by user pwilkin, aims to expand the model compatibility within the ggml-org ecosystem. The request …
-
Qwen 3.8 27B model available for local use with optimized KV cache
The Qwen 3.8 27B model, released by Alibaba Group, is now available for local execution, with specific GGUF file sizes detailed for various hardware configurations. A key feature is its hybrid attention mechanism, where…
-
llama.cpp adds CI targets for ROCm 7.14
A pull request has been submitted to the llama.cpp project to add Continuous Integration (CI) targets for ROCm 7.14. This update aims to enable the use of ROCm 7.14, which is the first production release utilizing the T…
-
llama.cpp adds MTP support for Qwen3-Next model
The open-source project llama.cpp has released version b10238, which includes Multi-Tentacle-Perception (MTP) support for the Qwen3-Next large language model. This update allows for more efficient local inference of Qwe…
-
llama.cpp integrates DSpark speculative decoding for performance gains
A pull request has been submitted to the llama.cpp project to integrate DSpark speculative decoding. This new feature aims to enhance performance by allowing the model to predict future tokens. The developers are encour…
-
llama.cpp sees performance boost via CUDA byte perm PR
A pull request has been submitted to the llama.cpp project on GitHub, titled "cuda: extract Q1_0 elements via __byte_perm". This change, submitted by dfriehs, aims to improve performance by extracting Q1_0 elements usin…
-
llama.cpp adds Q8_0 quantization support with ZenDNN backend, boosting performance
A pull request to the llama.cpp project introduces support for Q8_0 quantization within the ggml-zendnn backend. Benchmarks demonstrate significant performance gains, with ZenDNN_Q8_0 achieving up to a 193% speedup over…
-
llama.cpp releases multiple updates with cross-platform optimizations
The llama.cpp project has released several updates, including versions b10106, b10105, b10108, b10099, b10098, b10094, b10093, b10092, b10091, and b10103. These releases introduce various improvements and fixes across d…
-
Guide: Compile llama.cpp LLM runtime on Debian, cross-compile for ARM64
This technical guide details how to compile and cross-compile the llama.cpp project, a C/C++ based LLM inference runtime, on Debian Linux. It covers native builds for various architectures like x86-64 and ARM64, includi…
-
llama.cpp adds support for Tencent's Hy3 model with MTP speculative decoding
The llama.cpp project has integrated support for Tencent's Hy3 model, also known as hy_v3. This update includes the implementation of MTP speculative decoding, which allows for multi-token prediction. The Hy3 model is a…
-
llama.cpp adds ET backend for open-source hardware integration
The llama.cpp project has introduced a new backend called ggml-et, designed to support the ET-SOC-1 processor. This integration aims to incorporate open-source hardware into the broader open-source inference ecosystem. …
-
llama.cpp adds ARM NVFP4 dot product optimization via UE4M3 LUT
A pull request for the llama.cpp project introduces an optimization for ARM processors by extending the UE4M3 lookup table (LUT) to the NVFP4 dot product implementation. This change aligns the ARM version with existing …
-
llama.cpp adds -ffast-math flag for HIP builds, boosting performance
A pull request for the llama.cpp project introduces the ggml-hip library, enabling the use of the -ffast-math compiler flag for HIP builds. Benchmarks on an RDNA3.5 GPU show a performance increase of up to 7% for the Qw…
-
llama.cpp adds multi-layer MTP support via new pull request
A pull request has been submitted to the llama.cpp project to add support for Step3.5/3.7 flash MTP3. This update builds upon previous work and introduces multi-layer MTP support, encouraging users to try it with the la…
-
Georgi Gerganov praises Qwen3.6-27B for local coding tasks
Georgi Gerganov has found the Qwen3.6-27B model to be highly effective for local coding tasks, utilizing it daily for mundane maintenance work at ggml-org. He primarily runs the model on his Apple M2 Ultra or an RTX 509…
-
Cohere Labs releases North Mini Code for code generation
Cohere has released North Mini Code, a 30 billion parameter model optimized for code generation and agentic software engineering tasks. This open-weights research model, developed by Cohere and Cohere Labs, features a 3…