AVX2
PulseAugur coverage of AVX2 — every cluster mentioning AVX2 across labs, papers, and developer communities, ranked by signal.
2 day(s) with sentiment data
-
Developer prioritizes runtime checks over build configs for LLM binaries
A developer encountered a discrepancy between build system configurations and the actual runtime capabilities of a llama.cpp binary. Two sets of notes offered conflicting advice on enabling AVX2, with one suggesting it …
-
LLM inference on old hardware reveals evolving truths
The author details their experience running large language model inference on older hardware, drawing parallels to the evolving nature of scientific understanding. Initially, they held several assumptions about optimal …
-
llama.cpp adds AVX2 support for faster prompt processing
A pull request to the llama.cpp project introduces AVX2 instruction set support to accelerate prompt processing for IQ models, particularly with large batch sizes. This optimization aims to improve the speed of local la…
-
llama.cpp PR boosts IQ model prompt processing with AVX2 optimizations · 1 source tracked
A pull request for the llama.cpp project introduces AVX2 optimizations to significantly accelerate prompt processing for IQ models, particularly at large batch sizes. Benchmarks show dramatic speed increases, with some …
-
mistral.rs v0.9.0 achieves 1.8x faster CPU decode speeds than llama.cpp
The mistral.rs project has released version 0.9.0, demonstrating significant performance improvements in CPU decoding for large language models. Benchmarks show that mistral.rs can be up to 1.8 times faster than llama.c…
-
Intel launches Xeon 6+ with E-cores, drops hyper-threading
Intel has launched its new Xeon 6+ "Clearwater Forest" processors, featuring E-cores built on the 18A process and a higher core count. The company has deliberately omitted hyper-threading from these E-core server parts,…
-
StateSMix compressor uses Mamba SSMs and n-grams for online lossless compression
Researchers have developed StateSMix, a novel lossless compression algorithm that utilizes Mamba-style State Space Models (SSMs) combined with sparse n-gram context mixing. This system trains token-by-token on the data …
-
llama.cpp and ik_llama.cpp add FP4 inference support for VRAM savings
The llama.cpp and ik_llama.cpp projects have both integrated support for FP4 (4-bit floating-point) inference, a significant advancement for model quantization. llama.cpp now includes NVFP4, an Nvidia-specific format, w…
-
PHP-ORT brings machine learning inference to PHP developers
A new infrastructure project called PHP-ORT aims to bring machine learning inference capabilities directly to PHP, the server-side language used by a significant portion of the web. This development seeks to empower mil…