PulseAugur
中
实时 08:55:17
English(EN) NarrowSpec: Draft Hidden Width Need Not Match the Target in Speculative Decoding

新的NarrowSpec技术将大型语言模型解码速度提升高达1.44倍

研究人员开发了一种名为NarrowSpec的新技术,用于大型语言模型的推测解码。该方法允许用于验证的草稿模型以缩减的隐藏宽度(目标模型宽度的二分之一或四分之一)运行,而不会显著影响生成准确性。在Nvidia B200和DGX Spark硬件上使用SGLang对Qwen3-8B和GPT-OSS-20B模型进行的测试表明,NarrowSpec可以将端到端解码速度提高高达1.44倍,并且在各种批处理大小下都能保持加速效果。 AI

影响 加速大型语言模型的推理,可能降低生成式AI应用的成本并提高吞吐量。

排序理由 详细介绍大型语言模型推理优化新技术的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.LG 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →

新的NarrowSpec技术将大型语言模型解码速度提升高达1.44倍

本文如何被排名

Signal score
19 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Tool
详细介绍大型语言模型推理优化新技术的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration
Single-source cluster
Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics
paper, infra
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
Breaking (< 6h)
Fresh story with cross-source coverage still developing. Ranking may shift as more sources report.

完整方法见我们的编辑标准。

报道来源 [1]

  1. arXiv cs.LG TIER_1 English(EN) · Alagappan Valliappan, Ankit More, Dheevatsa Mudigere, Jatin Mitra ·

    NarrowSpec:推测式解码中草稿隐藏宽度无需与目标匹配

    arXiv:2611.23835v1 Announce Type: cross Abstract: Speculative decoding is widely used to speed up generation in large language models (LLMs), with a small draft model proposing tokens that one forward pass of the target model verifies. Draft models that read the target's hidden s…