研究人员开发了多种通过量化提高大语言模型（LLM）效率的新方法。OSAQ 专注于利用低秩 Hessian 属性抑制权重异常值，实现精确的低比特仅权重量化。BWLA 引入了一个框架，用于 1 位权重量化和低比特激活，实现了显著的推理加速。AGoQ 通过采用感知层激活量化和 8 位梯度存储，以内存高效的方式进行分布式训练，减少了内存使用并提高了训练速度。 AI
影响 大语言模型量化方面的这些进展有望显著降低计算成本和内存需求，从而实现更大模型的广泛部署和更快的推理。
排序理由 多篇 arXiv 论文介绍了用于大语言模型量化的新技术，重点关注效率和准确性改进。
AI 生成摘要 · Google Gemini · 来自 8 个来源。 我们如何撰写摘要 →
arXiv:2605.04738v1 Announce Type: new Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities. However, their massive parameter scale leads to significant resource consumption and latency during inference. Post-training weight-only quantization offers a p…
arXiv:2605.00422v1 Announce Type: new Abstract: Large language models (LLMs) have driven major progress in NLP, yet their substantial memory and compute demands still hinder practical deployment. Binarization can compress weights to 1 bit, fundamentally lowering compute and bandw…
arXiv:2605.00539v1 Announce Type: new Abstract: Quantization is a key method for reducing the GPU memory requirement of training large language models (LLMs). Yet, current approaches are ineffective for 4-bit activations and 8-bit gradients, which would easily cause slow converge…
Quantization is a key method for reducing the GPU memory requirement of training large language models (LLMs). Yet, current approaches are ineffective for 4-bit activations and 8-bit gradients, which would easily cause slow convergence or accuracy loss. To address this, we introd…
Large language models (LLMs) have driven major progress in NLP, yet their substantial memory and compute demands still hinder practical deployment. Binarization can compress weights to 1 bit, fundamentally lowering compute and bandwidth cost. However, existing methods cannot addr…
arXiv:2603.25385v2 Announce Type: replace-cross Abstract: Quantization techniques such as BitsAndBytes, AWQ, and GPTQ are widely used as a standard method in deploying large language models but often degrades accuracy when using low-bit representations, e.g., 4 bits. Low-rank cor…
arXiv:2605.00140v1 Announce Type: cross Abstract: We present Activation Residual Hessian Quantization (ARHQ), a post-training weight splitting method designed to mitigate error propagation in low-bit activation-weight quantization. By constructing an input-side residual Hessian f…
We present Activation Residual Hessian Quantization (ARHQ), a post-training weight splitting method designed to mitigate error propagation in low-bit activation-weight quantization. By constructing an input-side residual Hessian from activation quantization residuals (G_x), ARHQ …