EXL3
PulseAugur coverage of EXL3 — every cluster mentioning EXL3 across labs, papers, and developer communities, ranked by signal.
- 2026-06-20 product_launch A new method enables the conversion and inference of EXL3 quantized large language models on Apple Silicon Macs. source
4 day(s) with sentiment data
-
DeepSeek-V4.1-Flash-1017 gets major Delta Inference Engine update
The Delta Inference Engine has been updated to version 0.93, introducing significant improvements for the DeepSeek-V4.1-Flash-1017 model. This update enhances performance on a single RTX 3090 graphics card, offering fas…
-
New Gyro quantization method fragments community, lacks clear benefits over EXL3
A new quantization method called Gyro has emerged, drawing comparisons to the EXL3 quantization technique. While both utilize similar underlying technologies like QTIP and Hadamard rotations, Gyro employs a slightly sma…
-
EXL3 2.9bpw model's low KL divergence debated on r/LocalLLaMA
A discussion on Reddit's r/LocalLLaMA forum explores the technical merits of the EXL3 2.9bpw model, specifically its low Kullback–Leibler divergence. Users are questioning if this characteristic makes it superior to the…
-
GLM-5.3-Flash (321B) runs locally on single RTX 5090 via Project Maya
A technical guide details how to run the large GLM-5.3-Flash (321B) language model locally on a single RTX 5090 GPU with 128 GB of RAM. The process involves using Project Maya, a fork of the Strata runtime, which intell…
-
veloGB10 engine boosts Qwen3.8-Flash-Next performance on DGX Spark
The veloGB10, a new Rust/CUDA engine, has been tested with Qwen3.8-Flash-Next on DGX Spark hardware. This engine natively loads EXL3 weights and uses speculative decoding via MTP-Heads. Initial performance metrics show …
-
GLM-5.3-Flash achieves 384K context on dual CMP 170HX GPUs
A user has successfully configured and benchmarked the GLM-5.3-Flash model on a dual-GPU setup featuring two 64GB CMP 170HX cards. This setup utilizes the ExLlamaV3 inference engine and achieves a context window of 384K…
-
GLM-5.3-Flash EXL3 4bpw TensorFold Ablit model released with safety refusals removed
A new variant of the GLM-5.3-Flash model, specifically the EXL3 4bpw TensorFold Ablit version, has been released on Hugging Face. This model has had its safety refusals removed, making it suitable for research and unfil…
-
Qwen3.8 Flash Next model optimized for local inference on diverse hardware
The Qwen3.8 Flash Next model is being optimized for local inference across various hardware configurations. One user achieved 400 tokens/sec on an RTX 6000 with a fork of NInfer, while another demonstrated running the m…
-
New engine enables two 300B MoE models on single mini PCs
A new engine called Kyojin, built on ExLlamaV3 and optimized for AMD's Strix Halo hardware, has been developed to run large Mixture-of-Experts (MoE) models. This engine allows two 300B-class MoE models, GLM-5.3-Flash an…
-
ExLlamaSharp updates enhance local LLM server for Windows users
Kortexio has released updates for its ExLlamaSharp local LLM server, with versions v1.3.2.1, v1.3.2, and v1.3.1 detailing various improvements and fixes. These updates enhance the server's compatibility with NVIDIA GPUs…
-
EXL3 Quants Offer High Performance on Consumer Hardware
The EXL3 model, specifically its quantized versions, is being highlighted for its performance and efficiency on consumer hardware. Users report that EXL3 quants offer a significant speed advantage with minimal quality d…
-
ExLlamaSharp v1.2.1-beta adds OpenAI-compatible API and EXL3 inference
Kortexio has released ExLlamaSharp v1.2.1-beta, a local LLM server for Windows that supports NVIDIA GPUs. This beta version introduces OpenAI-compatible API endpoints, a Blazor admin interface, and enhanced EXL3 inferen…
-
EXL3 model loses traction in r/LocalLLaMA community despite performance
The EXL3 model, an alternative to llama.cpp for running large language models, appears to be losing traction within the r/LocalLLaMA community. While EXL3 offers strong performance in tokens-per-second and unique compre…
-
EXL3 LLM quants now convertible on Apple Silicon Macs
A new method allows for the conversion and inference of EXL3 quantized large language models on Apple Silicon Macs. Previously, these high-fidelity models were largely restricted to CUDA-enabled GPUs, requiring speciali…