English(EN) New quant types suggested: IQ2_K, IQ3_K, IQ4_K, IQ5_K and IQ6_K

为本地 LLM 提出新的 IQ-K 量化类型

作者 PulseAugur 编辑部 · [1 个来源] · 2026-10-18 10:06

为本地大型语言模型提出了一组新的量化类型，包括 IQ2_K、IQ3_K、IQ4_K、IQ5_K 和 IQ6_K。这些 IQ-K 量化旨在匹配现有 Q-quants 的文件大小，同时在 Kullback-Leibler 散度方面提供实质性改进。然而，这会带来轻微的性能下降和更慢的量化过程。 AI

影响引入了新的量化方法，可以提高在本地运行 LLM 的效率。

排序理由为 LLM 提出了新的量化类型，在 GitHub 拉取请求和 Hugging Face 模型中进行了详细介绍。 [lever_c_demoted from research: ic=1 ai=1.0]

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 1 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 为 LLM 提出了新的量化类型，在 GitHub 拉取请求和 Hugging Face 模型中进行了详细介绍。 [lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: infra; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: Same-day; Cluster formed today. Ranking reflects the current source set at time of score.

完整方法见我们的编辑标准。

r/LocalLLaMA TIER_1 English(EN) · /u/ethertype · 2026-10-18 10:06

新量化类型建议：IQ2_K、IQ3_K、IQ4_K、IQ5_K 和 IQ6_K

<div class="md">Looks quite interesting. Lots of numbers presented, but for a single model only. Highlights: <ul> <li>File sizes are the same as Q-quants. </li> <li>KLD improves substantially over Q-quants. </li> <li>Performance suffers slightly<…