两篇新研究论文介绍了用于加速大型语言模型推理的推测解码新方法。LibraSpec通过考虑接受额外标记的边际增益来优化推测长度，与自回归解码相比，速度提高了8.49倍。Goose提出了各向异性推测树，根据候选标记的质量自适应地构建它们，实现了1.9-4.3倍的无损加速，并优于平衡树基线。 AI
影响 这些技术可以显著降低大型语言模型的推理成本和延迟，使其在更广泛的应用中更易于访问和更高效。
排序理由 两篇在arXiv上发表的学术论文，详细介绍了LLM推测解码的新方法。
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
arXiv:2608.08721v1 Announce Type: cross Abstract: Speculative decoding accelerates large language model inference by drafting multiple tokens for parallel verification, with efficiency critically determined by the speculative length selected at each decoding round. Existing dynam…
arXiv:2604.02047v2 Announce Type: replace-cross Abstract: Speculative decoding accelerates large language model inference by drafting multiple candidate tokens and verifying them in a single forward pass. Candidates are organized as a tree: deeper trees accept more tokens per ste…