新研究探索用于效率的先进大模型量化技术

报道来源 [9]

1. arXiv cs.AI TIER_1 English(EN) · Sergii Kozyrev, Davyd Maiboroda · 2026-09-04 04:00

   Gated DeltaNet为何能承受4位量化：混合27B LLM的循环部分采用NVFP4 W4A4

   arXiv:2609.04098v1 Announce Type: new Abstract: Hybrid LLMs pair softmax attention with linear-attention layers such as Gated DeltaNet (GDN), whose recurrent state summarizes the context in fixed size. Early community 4-bit quantizations of Qwen3.8-27B (48 GDN layers, 16 attentio…

2. Hugging Face Daily Papers TIER_1 English(EN) · 2026-09-03 17:04

   Gated DeltaNet为何能承受4位量化：混合27B LLM的循环部分采用NVFP4 W4A4

   Hybrid LLMs pair softmax attention with linear-attention layers such as Gated DeltaNet (GDN), whose recurrent state summarizes the context in fixed size. Early community 4-bit quantizations of Qwen3.8-27B (48 GDN layers, 16 attention layers) left the GDN block in 8- or 16-bit pre…

3. arXiv cs.LG TIER_1 English(EN) · Xianghong Fang, Wenlong Mou, Yuan Yuan, Dehan Kong, Tim G. J. Rudner · 2026-09-03 04:00

   向量、乘积和标量量化的统一率失真视角

   arXiv:2609.02107v1 Announce Type: new Abstract: Discrete visual tokenization, predominantly driven by vector, scalar, and product quantization, lacks a unified conceptual framework for understanding quantization tradeoffs. In this paper, we propose a unified rate--distortion pers…

4. Hugging Face Daily Papers TIER_1 English(EN) · 2026-09-03 00:00

   Gated DeltaNet为何能承受4位量化：混合27B LLM的循环部分采用NVFP4 W4A4

   Fully quantizing hybrid LLMs—including recurrent Gated DeltaNet layers—to 4-bit NVFP4 preserves accuracy across long-context and reasoning benchmarks by localizing outliers and exploiting robust delta-rule dynamics.

5. arXiv cs.CL TIER_1 English(EN) · Jundong Hu, Shekar Ramachandran · 2026-09-02 04:00

   LLM量化损伤的结构：为何下一比特应全局分配

   arXiv:2609.01587v1 Announce Type: cross Abstract: Post-training quantization (PTQ) is widely used to reduce the cost of serving large language models (LLMs), but its accuracy cost is uneven and is often tuned per model. We study where quantization damage occurs and how to allocat…

6. arXiv cs.AI TIER_1 Italiano(IT) · Qian Zhang, Yaoming Li, Zhewen Tan, Yanshu Wang, Heng Lu, Kun Su, Zongwei Lv, Wenhan Yu, Yongge Ma, Yinjun Han, Ruikuang Liu, Tong Yang · 2026-09-02 04:00

   REAL-Q：通过动态梯度下降实现端到端大模型量化

   arXiv:2609.00049v1 Announce Type: cross Abstract: Post-training quantization (PTQ) is essential for deploying large language models (LLMs) under strict resource constraints. State-of-the-art PTQ methods quantize each layer with a single closed-form second-order solver: to remain …

7. arXiv cs.AI TIER_1 English(EN) · Chun-Ting Chen, Dongmin Han, Hangyeol Mun, Jake Hyun, Arnab Raha, Amit Agarwal, Mark Anders, Mohamed Abdelfattah, Jae-sun Seo · 2026-09-02 04:00

   HBQ：面向准确 LLM 推理的、具有硬件效率感知设计的层次化缩放块量化

   arXiv:2609.00450v1 Announce Type: cross Abstract: Block Quantization (BQ) is a promising approach for efficient deployment of large language models (LLMs), enabling low-precision computation with controlled accuracy degradation. Compared to scalar weight-only quantization (WoQ), …

8. arXiv cs.AI TIER_1 English(EN) · Deokjae Lee, Sihun Chu, Hyun Oh Song · 2026-09-01 04:00

   Q-Strata：混合精度量化MoE大模型的层级比特分配

   arXiv:2608.30564v1 Announce Type: cross Abstract: Mixed-precision quantization (MPQ) assigns a different bitwidth to each linear layer of a large language model (LLM) to minimize the quantization-induced quality loss under a fixed budget, but Mixture-of-Experts (MoE) models conta…

9. arXiv cs.LG TIER_1 English(EN) · Artem Safronov · 2026-08-31 04:00

   通过性能最大化和质量下降约束的层比特宽度分配方法用于LLM量化

   arXiv:2608.28003v1 Announce Type: new Abstract: This paper proposes a layer bit allocation method for Gemma-3-1B, formulating the problem as performance maximization (latency decrease) given a degradation budget constraint (allowable level of generation quality loss). This approa…