PulseAugur
中
实时 18:37:44

新方法通过推测性解码加速大语言模型推理 · 跟踪10个来源

研究人员正在开发新的方法,通过推测性解码来加速大语言模型(LLM)的推理。这些技术包括使用一个轻量级的“草稿”模型并行生成多个标记,然后由一个更大的“目标”模型进行验证。S$^3$R、DCOCO 和 Nucleus Speculative Decoding 等新方法旨在通过优化验证过程、恢复上下文依赖关系或允许更灵活的接受规则来提高效率。一些方法还探索分析 LLM 的隐藏状态以更好地选择用于验证的层,而另一些方法则侧重于防御会降低推测性解码性能的攻击。 AI

影响 推测性解码的这些进展可以显著降低 LLM 推理成本和延迟,从而实现更广泛的部署和更具响应性的 AI 应用。

排序理由 多篇 arXiv 上的研究论文详细介绍了 LLM 中推测性解码的新方法。

在 arXiv cs.CL 阅读 →

AI 生成摘要 · Google Gemini · 来自 22 个来源。 我们如何撰写摘要 →

新方法通过推测性解码加速大语言模型推理 · 跟踪10个来源

本文如何被排名

Signal score
0 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Research
多篇 arXiv 上的研究论文详细介绍了 LLM 中推测性解码的新方法。
Source corroboration
22 independent sources
Strong cross-source corroboration — multiple independent publishers covered this within the clustering window.
Topics
infra, paper
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
47 days old
Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.
Coverage growth since scoring
+3 source(s) since last score
New sources have picked up this story since our last re-score. Score will update on the next scoring pass.

完整方法见我们的编辑标准。

报道来源 [22]

  1. arXiv cs.CL TIER_1 English(EN) · Hanbin Bae, Heejin Choi, Joohyung Lee, Byoung Jin Choi, Sangjun Park, Hoon-Young Cho ·

    HyCo:混合草稿用于成本高效的移动云推断式解码

    arXiv:2610.21436v1 Announce Type: new Abstract: Device-cloud speculative decoding reduces server computation, but growing a deep draft tree with a small language model (SLM) requires repeated passes on a mobile neural processing unit (NPU). We present HyCo, which assigns depth to…

  2. arXiv cs.CL TIER_1 English(EN) · Jihao Zhao, Qianyu Jiang, Shuaishuai Zu, Pengfei Li, Guanghuan Fang, Ben Feng, Biao Qin, Zhibin Wang, Baochuan Yang ·

    TreeSpark:学习起草您用于推测性解码的树

    arXiv:2610.20175v1 Announce Type: new Abstract: Parallel drafters such as DFlash and DSpark reduce proposal latency, but early mismatches limit how much of each block can be accepted. Retaining alternatives introduces parent states and candidate choices that reference-conditioned…

  3. arXiv cs.CL TIER_1 English(EN) · Shuxuan Li, Jialiang Kang, Yanghao Zhou, Ziye Zhu, Xiao Yang ·

    FOLD:用于长上下文投机解码的无位置局部排序远记忆

    arXiv:2610.20392v1 Announce Type: new Abstract: Speculative decoding accelerates LLM generation through lightweight drafting and parallel verification. It is particularly promising for long inputs, where KV-cache access costs can be amortized over multiple accepted tokens. Yet mo…

  4. arXiv cs.AI TIER_1 English(EN) · Naji Alazhar, Wassel Alazhar ·

    Adaptive DHiSparse: 基于负载自适应KV层次结构的推测式解码

    arXiv:2610.13708v1 Announce Type: new Abstract: Long-context serving runs out of KV memory before the hardware reaches its useful throughput. HiSparse stores history in host memory and fetches only the selected entries, but it uses one cache budget at every load and does not hand…

  5. arXiv cs.AI TIER_1 English(EN) · Raghavv Goel, Sudhanshu Agrawal, Mukul Gagrani, Risheek Garrepalli, Mingu Lee ·

    冲浪激活浪潮:分析LLM隐藏状态的信息与几何以进行推测性解码

    arXiv:2610.12939v1 Announce Type: cross Abstract: Decoder hidden states from different layers of large language models (LLMs) often encode complementary information. However, existing speculative decoding methods typically rely on a small set of target-model layers chosen using f…

  6. arXiv cs.CL TIER_1 English(EN) · Yuhang He, Hongyu Ge, Zihan Lin, Siyi Liu, Yongqi Zhang ·

    S$^3$R:通过自稀疏推测解码实现快速忠实的 RL 回滚

    arXiv:2610.13202v1 Announce Type: cross Abstract: Fast rollout generation for long-context reinforcement learning with verifiable rewards (RLVR) requires reducing KV-cache traffic without changing the response groups used for policy optimization. In our profiled RLVR workloads, a…

  7. arXiv cs.CL TIER_1 English(EN) · Roy Eisenstadt, Harel Lidar, Tal Gonen, Omer Luxembourg, Itamar Zimerman, Lior Wolf, Eli David, Ilan Naiman ·

    Contextual Correction of Parallel Drafts for Speculative Decoding

    arXiv:2610.12570v1 Announce Type: new Abstract: Speculative decoding accelerates large language model inference by using a lightweight drafter to propose multiple tokens that are subsequently verified in parallel by the target model. Recent diffusion-based drafters make this para…

  8. arXiv cs.AI TIER_1 English(EN) · Shijing Hu, Xuancheng Ren, Zhihui Lu, Pan Zhou ·

    草稿树失去目标质量的地点:出口引导的投机解码

    arXiv:2610.11750v1 Announce Type: new Abstract: Tree-based speculative decoding verifies multiple draft continuations in one target-model pass, but finite trees built from draft scores face a fundamental draft-target mismatch. We ask whether better exact verification can increase…

  9. arXiv cs.AI TIER_1 English(EN) · Adam Y. J. Jones, Yu Yuan, Sergio Maffeis ·

    Speedbumps:对推测解码的拒绝攻击

    arXiv:2610.10929v1 Announce Type: cross Abstract: Speculative decoding is a popular technique for increasing the speed and reducing the costs of large language model (LLM) inference by verifying multiple draft tokens in a single target-model forward pass. The resulting benefit de…

  10. Hugging Face Daily Papers TIER_1 English(EN) ·

    冲浪激活浪潮:分析LLM隐藏状态的信息与几何以进行推测性解码

    Decoder hidden states from different layers of large language models (LLMs) often encode complementary information. However, existing speculative decoding methods typically rely on a small set of target-model layers chosen using fixed, position-based heuristics. Selecting an opti…

  11. arXiv cs.CL TIER_1 English(EN) · Yunxiao Zhao, Changxiao Cai ·

    通过直接最小化预期解码轮次来训练并行推测草稿模型

    arXiv:2610.10411v1 Announce Type: cross Abstract: Speculative decoding accelerates large language model inference by using a low-cost draft model to propose tokens that the full-size target model verifies in parallel. Parallel and semi-autoregressive (semi- AR) drafters improve d…

  12. Hugging Face Daily Papers TIER_1 English(EN) ·

    Contextual Correction of Parallel Drafts for Speculative Decoding

    Speculative decoding accelerates large language model inference by using a lightweight drafter to propose multiple tokens that are subsequently verified in parallel by the target model. Recent diffusion-based drafters make this paradigm particularly attractive: deep attention-bas…

  13. arXiv cs.AI TIER_1 English(EN) · Chung-En Ho, Weiyu Sun, Cheng-Jhih Shih, He Li, Yong Liu, Yingyan Celine Lin ·

    SpecFold:折叠多分支冗余以加速扩散语言模型中的推测性解码

    arXiv:2610.04875v2 Announce Type: replace Abstract: Diffusion large language models (DLLMs) generate text through iterative block denoising, and multi-branch speculative decoding accelerates this process by verifying a main branch together with multiple draft branches in a single…

  14. arXiv cs.AI TIER_1 English(EN) · Yichi Zhang, Zhiqi Wang, Neil Gong, Yuchen Yang ·

    面向大型语言模型的安全推测解码

    arXiv:2610.08678v1 Announce Type: cross Abstract: Speculative decoding accelerates inference for a large language model (LLM), referred to as the \emph{target model}, by first using a smaller model, referred to as the \emph{draft model}, to generate candidate tokens and then veri…

  15. arXiv cs.CL TIER_1 English(EN) · Geonmo Gu, Byeongho Heo, HeeJae Jun, Yoohoon Kang, Sangmin Lee, Sangdoo Yun, Dongyoon Han ·

    DLoop:循环推断解码

    arXiv:2610.07659v1 Announce Type: new Abstract: Speculative decoding accelerates autoregressive generation in large language models. In each drafting stage, a lightweight draft model proposes tokens that the target model subsequently verifies. With increasingly capable draft mode…

  16. arXiv cs.CL TIER_1 English(EN) · Sumin Lee, Sukmin Cho, Seungjae Lim, Youngjin Kwon ·

    AgSpec:在编码代理管道中推动检索式推测解码的极限

    arXiv:2610.01108v2 Announce Type: replace Abstract: Retrieval-based speculative decoding (SD) drafts tokens by copying continuations from existing text, which suits coding agents that repeatedly reproduce code, logs, and earlier attempts. Yet existing methods fall short in agent …

  17. arXiv cs.CL TIER_1 English(EN) · Shuhao Li, Fanghua Ye, Wanyu Lin, Tianyu Yuan, Xiaoyu Shen ·

    Nucleus Speculative Decoding: Plausibility-Aware Verification Beyond Exact Distribution

    arXiv:2610.07822v1 Announce Type: new Abstract: Speculative decoding accelerates autoregressive generation by using a lightweight draft model to propose multiple tokens that are verified by a target model in parallel. However, the standard acceptance rule focuses on exact distrib…

  18. Hugging Face Daily Papers TIER_1 English(EN) ·

    SpecFold:折叠多分支冗余以加速扩散语言模型的推测性解码

    Diffusion large language models (DLLMs) generate text through iterative block denoising, and multi-branch speculative decoding accelerates this process by verifying a main branch together with multiple draft branches in a single forward pass. While prior DLLM acceleration methods…

  19. Hugging Face Daily Papers TIER_1 English(EN) ·

    DLoop:循环推断解码

    Speculative decoding accelerates autoregressive generation in large language models. In each drafting stage, a lightweight draft model proposes tokens that the target model subsequently verifies. With increasingly capable draft models, we find that the target model frequently acc…

  20. r/LocalLLaMA TIER_1 English(EN) · /u/pmttyji ·

    [论文] DLoop:循环推断解码

    <table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1x1eu7o/paper_dloop_looped_speculative_decoding/"> <img alt="[Paper] DLoop: Looped Speculative Decoding" src="https://preview.redd.it/qt7fjeyd7euh1.png?width=140&amp;height=72&amp;auto=webp&amp;s=28336cf2022f4…

  21. dev.to — LLM tag TIER_1 English(EN) · jidonglab ·

    Speculative Decoding 让我的 vLLM 服务器变慢了:接受度数学

    <p>I added a draft model to my vLLM server, sent one curl request, and watched the tokens fly. Then I ran the load test with 64 concurrent users and total throughput went <em>down</em>. Same GPU, same target model, same prompts. The only change was one flag that every blog post s…

  22. r/LocalLLaMA TIER_1 English(EN) · /u/pmttyji ·

    [论文] WaveFront Decoding:用于循环语言模型的并行化自推测解码

    <table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1wz2e0o/paper_wavefront_decoding_parallelized/"> <img alt="[Paper] WaveFront Decoding: Parallelized Self-Speculative Decoding for Looped Language Models" src="https://preview.redd.it/3agb0qbdkuth1.png?width=14…