SmoothQuant
PulseAugur coverage of SmoothQuant — every cluster mentioning SmoothQuant across labs, papers, and developer communities, ranked by signal.
2 day(s) with sentiment data
-
LLM Quantization Failure: Outliers Zero Out 99.9% of Data
A technical blog post details a critical failure mode in INT4 and INT8 quantization for large language models (LLMs) caused by outliers. When a single outlier value is present in a block of 4,096 floating-point numbers,…
-
Beginner's guide to LLM quantization resources compiled
A curated list of resources for beginners interested in quantization for large language models has been compiled. The list includes key papers and techniques such as BitNet, AWQ, QLoRA, and ZeroQuant, covering topics fr…
-
New LLM quantization methods promise reduced memory and faster inference · 2 sources tracked
Two new research papers introduce novel methods for quantizing large language models (LLMs) to reduce their memory footprint and improve inference speed. SpectrumQuant proposes a data-aware orthogonal transform method t…
-
New ADMM-Q algorithm enhances LLM quantization, reducing perplexity
Researchers have developed ADMM-Q, a new algorithm designed to improve post-training quantization for large language models. This method utilizes a combinatorial variant of the Alternating Direction Method of Multiplier…
-
New research explores advanced LLM quantization techniques for efficiency
Several new research papers explore advanced techniques for quantizing large language models (LLMs) to improve efficiency for deployment. REAL-Q introduces a dynamic gradient descent method to minimize end-to-end KL div…
-
LLM Quantization: More Than Just Bit Reduction
Quantization in large language models is a complex process involving more than just reducing bit precision. It encompasses four key decisions: notation, format, evaluation, and the resulting capacity gains. Different qu…
-
New preconditioning method boosts quantized matrix multiplication accuracy
Researchers have developed a new method called Contraction-Gauge Preconditioning to improve the accuracy of quantized matrix multiplication, a key operation in deep learning. This technique jointly selects a factor repr…
-
New W4A4 quantization technique enhances Wan2.2-I2V-A14B model inference
Researchers have developed a novel W4A4 quantization technique for the Wan2.2-I2V-A14B model, aiming to improve inference efficiency on low-bit-width hardware. Their approach combines mixed precision for activation outl…
-
OpenPangu LLM quantization on Ascend NPUs shows 8-bit is lossless, 4-bit degrades 1B model
A new study investigates the effectiveness of various post-training quantization methods for the OpenPangu large language models when deployed on Ascend NPUs. Researchers found that 8-bit weight-only quantization is nea…
-
llmcompressor tool enables LLM compression via FP8, GPTQ, SmoothQuant
A new open-source tool named llmcompressor allows developers to compress and benchmark instruction-tuned large language models. The tool demonstrates how to apply post-training quantization techniques such as FP8, GPTQ,…
-
Optimizing Transformer Inference: Techniques for Faster, Cheaper Large Models
Large transformer models present significant inference challenges due to their substantial memory footprint and computation costs, which scale quadratically with input length. Researchers and practitioners are exploring…