研究人员推出了一种用于大语言模型（LLM）混合精度量化 的新方法 CoQuant。该技术通过联合考虑权重和激活统计信息来识别高精度保留的关键子空间，从而解决了现有方法的局限性。CoQuant 利用理论建模的误差和加权 PCA 解决方案来平衡这些协方差，旨在更有效地降低推理成本。在 Llama-3.2 和 Qwen2.5 模型上的实验表明，与当前的训练后量化基线相比，CoQuant 在困惑度和推理准确性方面表现更优。 AI
影响 通过优化的混合精度量化降低推理成本，从而提高大语言模型的效率。
排序理由 该集群包含一篇详细介绍大语言模型量化新方法的学术论文。
AI 生成摘要 · Google Gemini · 来自 3 个来源。 我们如何撰写摘要 →
arXiv:2604.26378v1 Announce Type: new Abstract: Post-training quantization (PTQ) has become an important technique for reducing the inference cost of Large Language Models (LLMs). While recent mixed-precision methods improve ultra-low bit quantization by preserving critical subsp…
Post-training quantization (PTQ) has become an important technique for reducing the inference cost of Large Language Models (LLMs). While recent mixed-precision methods improve ultra-low bit quantization by preserving critical subspaces in high precision, they typically construct…
Post-training quantization (PTQ) has become an important technique for reducing the inference cost of Large Language Models (LLMs). While recent mixed-precision methods improve ultra-low bit quantization by preserving critical subspaces in high precision, they typically construct…