W4A16
PulseAugur coverage of W4A16 — every cluster mentioning W4A16 across labs, papers, and developer communities, ranked by signal.
-
LLM Quantization: More Than Just Bit Reduction
Quantization in large language models is a complex process involving more than just reducing bit precision. It encompasses four key decisions: notation, format, evaluation, and the resulting capacity gains. Different qu…
-
ExactMoE slashes MoE model memory use by 87% with minimal accuracy loss
Researchers have developed ExactMoE, a novel inference design for sparse mixture-of-experts (MoE) language models that significantly reduces memory requirements. By applying four-bit weight quantization only to the acti…
-
INT4 Weight-Only Quantization: Decode Speedup, Prefill Stagnation Explained
Weight-only INT4 quantization, while effective for reducing memory traffic and speeding up the decoding phase of LLM inference, does not improve the prefill phase. This is because prefill is compute-bound, meaning it is…