ExLlamaV3
PulseAugur coverage of ExLlamaV3 — every cluster mentioning ExLlamaV3 across labs, papers, and developer communities, ranked by signal.
- 2026-07-15 product_launch ExLlamaV3 has released version 1.0.0, introducing significant performance upgrades and new features. source
4 day(s) with sentiment data
-
Developer probes reveal misleading batch limits in LLM inference servers
A developer investigated the actual batch processing capabilities of an inference server, finding that advertised batch limits did not reflect real-world performance. Through custom probes and code analysis, it was disc…
-
Four large local LLMs benchmarked on RTX 5090 with 128GB RAM
A technical benchmark compares four large local language models (LLMs) running on a single RTX 5090 GPU with 128 GB of RAM. The models tested include DeepSeek-V4.1-Flash (748B parameters), GLM-5.3-Flash (321B parameters…
-
GLM-5.3-Flash (321B) runs locally on single RTX 5090 via Project Maya
A technical guide details how to run the large GLM-5.3-Flash (321B) language model locally on a single RTX 5090 GPU with 128 GB of RAM. The process involves using Project Maya, a fork of the Strata runtime, which intell…
-
763B MoE model runs on dual 96GB GPUs via expert offload
A technical guide details how to run the 763 billion parameter DeepSeek-V4.1-Flash model on consumer-grade hardware, specifically two Nvidia RTX PRO 6000 Blackwell GPUs with 96GB of VRAM each. The solution involves offl…
-
Gemma E4B optimized for Jetson Orin Nano with new EXL3 engine
A new engine called little-gemma V1.0 has been developed for the NVIDIA Jetson Orin Nano, offering improved performance over llama.cpp for the Gemma E4B model. The developer forked exllamav3 and ported code from little-…
-
GLM-5.3-Flash achieves 384K context on dual CMP 170HX GPUs
A user has successfully configured and benchmarked the GLM-5.3-Flash model on a dual-GPU setup featuring two 64GB CMP 170HX cards. This setup utilizes the ExLlamaV3 inference engine and achieves a context window of 384K…
-
Qwen3.8 Flash Next model optimized for local inference on diverse hardware
The Qwen3.8 Flash Next model is being optimized for local inference across various hardware configurations. One user achieved 400 tokens/sec on an RTX 6000 with a fork of NInfer, while another demonstrated running the m…
-
GLM-5.3-UNCENSORED EXL3 3.0bpw model released with ExLlamaV3 quantization
A quantized version of the GLM-5.3 model, named GLM-5.3-UNCENSORED EXL3 3.0bpw, has been released on Hugging Face. This model is based on the dealignai/GLM-5.3-UNCENSORED-FP8, which itself is a weight-edited variant of …
-
ExLlamaSharp updates enhance local LLM server for Windows users
Kortexio has released updates for its ExLlamaSharp local LLM server, with versions v1.3.2.1, v1.3.2, and v1.3.1 detailing various improvements and fixes. These updates enhance the server's compatibility with NVIDIA GPUs…
-
ExLlamaV3 praised for superior performance in local LLM deployment
A Reddit user is advocating for ExLlamaV3, a software that they believe is underrated for running large language models locally. They highlight its superior quantization quality, lower KLD metrics, and faster performanc…
-
ExLlamaV3 shows significant speed gains over llama.cpp for Qwen model
A user on Reddit's r/LocalLLaMA community shared their experience with ExLlamaV3, reporting significantly faster performance compared to llama.cpp when running the Qwen-3.8-Flash-Next model. The user observed a 3.2x inc…
-
ExLlamaV3 praised for impressive speed and performance
A user on Reddit shared their positive experience with ExLlamaV3, a new model they tested. They reported impressive speeds of 700 tokens/second for prefill and 42 tokens/second for decoding when running GLM 5.3 Flash on…
-
ExLlamaV3 v1.0.0 released with major performance upgrades
The ExLlamaV3 project has released version 1.0.0, marking a significant performance upgrade after over a year of development. This release introduces a new attention kernel with advanced quantization and caching, improv…
-
ExLlamaV3, Unsloth Qwen, and Phi3 agent see major local AI updates
This week's local AI news highlights significant updates to the ExLlamaV3 inference library, enhancing efficiency for running quantized Llama models on consumer GPUs. Additionally, new GGUF-quantized versions of Qwen 3.…