New MARR technique boosts low-bit quantization for LLMs and ViTs

Researchers have developed a new technique called Module-Adaptive Residual Reconstruction (MARR) to improve low-bit post-training quantization for large language models and vision transformers. MARR addresses limitations in existing methods by adaptively balancing error correction and bias across different model modules. This approach uses a module-specific scaling coefficient and a PID-based update strategy to refine coefficients, leading to significant performance gains, particularly at quantization levels of 4-bit or lower. AI

影响 Enhances efficiency of LLMs and ViTs by improving low-bit quantization techniques.

排序理由 Academic paper detailing a new method for model quantization. [lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.CV 阅读 →

• LLMs
• MARR
• ViTs

• 论文
• 模型发布

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →