PulseAugur
中
实时 16:15:27

llama.cpp 发布提升 Zamba2 性能，优化 MoE 缓存

作者 PulseAugur 编辑部 · [4 个来源] ·

llama.cpp 项目发布了多个预发布版本，包括 b12207，通过优化状态大小为 64 的 SSM_SCAN 内核，显著提升了 Zamba2 类架构的性能。此优化使 Zamba2-7B-Instruct 在 GPU 上的吞吐量提高了约 35%，尽管一些扫描调用仍回退到 CPU，原因尚待确定。其他近期版本，如 b12206 和 b12201，解决了 MoE 缓存和推测解码的问题，确保了在 macOS、iOS、Linux、Android 和 Windows 等各种平台上的效率和稳定性。 AI

影响 提高消费级硬件上各种 LLM 架构的推理速度和稳定性。

排序理由 开源推理库的多个预发布更新，包括性能优化和错误修复。

在 llama.cpp — Releases 阅读 →

AI 生成摘要 · Google Gemini · 来自 4 个来源。 我们如何撰写摘要 →

llama.cpp 发布提升 Zamba2 性能，优化 MoE 缓存

本文如何被排名

Signal score
6 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Research
开源推理库的多个预发布更新，包括性能优化和错误修复。
Source corroboration
4 independent sources
Strong cross-source corroboration — multiple independent publishers covered this within the clustering window.
Topics
infra, product
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
Same-day
Cluster formed today. Ranking reflects the current source set at time of score.

完整方法见我们的编辑标准。

报道来源 [4]

  1. llama.cpp — Releases TIER_1 (SO) · github-actions[bot] ·

    b12207

    <details open=""> <p>ggml-cuda : add d_state=64 fast path to SSM_SCAN kernel (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/31818">#31818</a>)</p> <p>Zamba2 and similar architectures use a state size of 64 in their<br /> recurrent-scan step,…

  2. llama.cpp — Releases TIER_1 (SO) · github-actions[bot] ·

    b12206

    <details open=""> <p>moe-cache : ensure CPU part does not wait for the cached experts (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/31888">#31888</a>)</p> </details> <p><strong>Website:</strong></p> <ul> <li><a href="https://llama.app" rel=…

  3. llama.cpp — Releases TIER_1 (SO) · github-actions[bot] ·

    b12202

    <details open=""> <p>llama, mtmd: follow the reference precision of ggml_prec_set_acc calls (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/31851">#31851</a>)</p> <ul> <li>llama, mtmd: follow the reference precision of ggml_prec_set_acc calls…

  4. llama.cpp — Releases TIER_1 (SO) · github-actions[bot] ·

    b12201

    <details open=""> <p>speculative : do not apply the MoE cache to the draft model (<a class="issue-link js-issue-link" href="https://github.com/ggml-org/llama.cpp/pull/31497">#31497</a>)</p> </details> <p><strong>Website:</strong></p> <ul> <li><a href="https://llama.app" rel="nofo…

相关实体

相关话题