研究人员开发了CAT-Q，一种新颖的训练后量化方法，可在无需大量重新训练的情况下显著压缩和加速大语言模型（LLMs）。该技术采用可学习调制和软化三值化，能够高效地量化从1.7B到235B参数的模型，仅使用少量校准样本，并取得了优于BitNet等现有方法的性能。此外，GRINQH提供了一种基于梯度输入的量化层级，通过根据激活值大小动态分配精度级别来优化大语言模型的生成，在Llama 3和Qwen3等模型上表现优于当前基线。 AI
影响 这些量化方面的进展可能显著降低部署大语言模型的计算成本和内存需求，使其在各种应用中更易于访问，尤其是在资源受限的环境中。
排序理由 两篇研究论文详细介绍了用于大语言模型量化（quantization）的新方法。
AI 生成摘要 · Google Gemini · 来自 4 个来源。 我们如何撰写摘要 →
arXiv:2606.26650v1 Announce Type: cross Abstract: In this paper, we present CAT-Q, Cost-efficient and Accurate Ternary Quantization, for compressing and accelerating LLMs. Unlike existing state-of-the-art ternary quantization methods that rely on data-intensive and costly quantiz…
In this paper, we present CAT-Q, Cost-efficient and Accurate Ternary Quantization, for compressing and accelerating LLMs. Unlike existing state-of-the-art ternary quantization methods that rely on data-intensive and costly quantization-aware training to mitigate severe performanc…
In this paper, we present CAT-Q, Cost-efficient and Accurate Ternary Quantization, for compressing and accelerating LLMs. Unlike existing state-of-the-art ternary quantization methods that rely on data-intensive and costly quantization-aware training to mitigate severe performanc…
Autoregressive decoding with LLMs is primarily bottlenecked by GPU memory bandwidth, especially in edge-computing settings. While quantization is essential for mitigating this bottleneck, most existing methods treat inference as a uniform process and fail to account for the asymm…