English(EN) Qwen3.5 0.8B on CPU

Qwen 3.5 0.8B 模型通过新的量化格式针对 CPU 进行了优化

作者 PulseAugur 编辑部 · [1 个来源] · 2026-09-07 20:09

一位开发者为 Qwen 3.5 0.8B 模型创建了一个自定义 C++ 引擎和一种新的 4 位量化格式 H128/Q4-G32-DOT4。这种新格式使模型大小减小到 425 MB，比 Unsloth 的混合精度 Q4_0 小 71 MB。在 Ryzen 9 9955HX3D CPU 上的基准测试显示性能显著提升，自定义引擎的预填充速度最高可达 2.9 倍，批处理 16 的吞吐量比其他引擎高 1.7 倍。 AI

影响使得在消费级硬件上更高效地本地部署小型语言模型成为可能。

排序理由针对现有模型的自定义引擎和量化格式。

在 r/LocalLLaMA 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 0 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 针对现有模型的自定义引擎和量化格式。
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: infra, model release; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: 2 days old; Coverage has settled into its steady-state source set.

完整方法见我们的编辑标准。

报道来源 [1]

r/LocalLLaMA TIER_1 English(EN) · /u/Danmoreng · 2026-09-07 20:09

Qwen3.5 0.8B on CPU

<div class="md"><p>Since the Qwen3.5 0.8B model is an interesting one for small specialized fine tunes, I was curious how fast it can run on CPUs. Why CPUs? Mainly because I want to use it as a local dictation cleanup model when I'm using the GPU for something else…

本文如何被排名

报道来源 [1]

Qwen3.5 0.8B on CPU

相关实体

相关话题