GPTQ
PulseAugur coverage of GPTQ — every cluster mentioning GPTQ across labs, papers, and developer communities, ranked by signal.
- competes with Activation Aware Quantization 75%
- instance of Activation Aware Quantization 70%
- instance of GGUF 70%
- used by waqf 70%
- used by vLLM 70%
- instance of Gotit.pub 70%
- uses vLLM 70%
- instance of bfloat16 70%
- used by GGUF 60%
- used by Activation Aware Quantization 60%
- uses GGUF 60%
- used by Int4 60%
5 day(s) with sentiment data
-
New WaterKron method improves AI model quantization using Kronecker-factored Hessians
Researchers have developed WaterKron, a novel method for post-training quantization that utilizes Kronecker-factored Hessian approximations. This approach combines two-sided GPTQ with waterfilling scales and entropy cod…
-
Qwen3-8B model scaled for ultra-low-bit language processing
Researchers have successfully scaled post-training ternarisation techniques to the Qwen3-8B language model, aiming to reduce storage and memory requirements. The study involved a comprehensive evaluation, including repr…
-
Mistral Small 3.2 released with enhanced function calling and 128K context
Mistral AI has released Mistral Small 3.2, an open-weight model featuring improved function calling and a 128K context window. This update enhances the model's ability to handle tool distinctions and provides cleaner JS…
-
New ADMM-Q algorithm enhances LLM quantization, reducing perplexity
Researchers have developed ADMM-Q, a new algorithm designed to improve post-training quantization for large language models. This method utilizes a combinatorial variant of the Alternating Direction Method of Multiplier…
-
New research explores advanced LLM quantization techniques for efficiency
Several new research papers explore advanced techniques for quantizing large language models (LLMs) to improve efficiency for deployment. REAL-Q introduces a dynamic gradient descent method to minimize end-to-end KL div…
-
Quantized LLMs Exhibit Layer-Dependent Personalities, Study Finds
A new research paper explores the personality traits of quantized large language models (LLMs), moving beyond previous studies that focused only on full-precision models. The study introduces Uncertainty-Amplified Layer…
-
Guide Explains VRAM Needs for Local LLM Deployment
Running large language models locally requires careful VRAM management, as model size and quantization significantly impact memory usage. While there's no exact formula, VRAM needs can be estimated by considering the mo…
-
LLM Quantization: More Than Just Bit Reduction
Quantization in large language models is a complex process involving more than just reducing bit precision. It encompasses four key decisions: notation, format, evaluation, and the resulting capacity gains. Different qu…
-
SchurQuant enhances LLM quantization accuracy, improving 2-bit models by over 11 pp
Researchers have developed SchurQuant, a novel method for layer-wise quantization of large language models (LLMs) that significantly improves accuracy, especially at lower bitrates. The technique, which includes a compo…
-
New methods refine LLM quantization for reduced size and cost · 2 sources tracked
Two new research papers propose novel methods for post-training quantization (PTQ) of large language models, aiming to reduce their size and computational requirements. The first paper, "From Sweep to Seam: Interleaved …
-
New KLQ quantization method optimizes LLM bit-width allocation
A new research project, KLQ, introduces a training-free method for quantizing large language models. This approach measures the unevenness of embedding spaces and optimally allocates bit-widths to different directions b…
-
LLM users seek guidance on comparing quantized models from diverse sources
Users on the r/LocalLLaMA subreddit are seeking guidance on how to effectively compare and manage quantized large language models (LLMs) from various sources. The primary challenge lies in the overwhelming number of var…
-
New GRIT format aims to prevent silent bugs in quantized AI models
A new interchange format called GRIT (Grouped Reduced-precision Interchange Type) has been developed to address critical bugs in quantized model checkpoints. These bugs, which occurred silently in systems like vLLM and …
-
New methods improve neural network quantization efficiency and accuracy
Researchers have developed new methods for neural network quantization, a process that reduces the memory and computational requirements of AI models. The first paper introduces BaKron, an efficient solver that uses Kro…
-
New research explores LLM efficiency and reasoning improvements
Several research papers explore methods to enhance the efficiency and reliability of large language models (LLMs). Hugging Face's LFM2.5-DSpark demonstrates up to 3.2x faster inference speeds by using speculative decodi…
-
AI Development Shifts Local-First by 2026 for Speed and Privacy
The AI development landscape is rapidly shifting towards a local-first approach, driven by the need to overcome cloud API latency, ensure data privacy, and reduce costs. By 2026, running AI models on local hardware is e…
-
New GPTQ-2D algorithm slashes matrix rounding time to cubic
Researchers have introduced GPTQ-2D, a novel algorithm designed to improve the efficiency of adaptive rounding methods for matrices. Unlike existing methods that process matrix entries sequentially, GPTQ-2D rounds entri…
-
GPTQ-2D advances matrix rounding with cubic-time parallel processing
Researchers have introduced GPTQ-2D, a novel method for adaptive rounding of real matrices to integers. This technique, an extension of existing methods like GPTQ and Babai's nearest plane algorithm, addresses the two-s…
-
AWQ outperforms GPTQ in 4-bit quantization for local LLMs, but GPU and kernels are key
A comparison of 4-bit quantization methods for local Large Language Models (LLMs) indicates that Activation Aware Quantization (AWQ) generally outperforms GPTQ. However, the study emphasizes that the actual performance …
-
Quantization impacts code generation models differently, study finds
A new study investigates the impact of various quantization methods on the performance of large code generation models when run on resource-constrained hardware. Researchers evaluated six state-of-the-art techniques, in…