PulseAugur
EN
LIVE 17:04:47

New methods accelerate LLM inference with speculative decoding · 10 sources tracked

Researchers are developing new methods to accelerate large language model (LLM) inference through speculative decoding. These techniques involve using a lightweight "drafter" model to propose multiple tokens in parallel, which are then verified by a larger "target" model. New approaches like S$^3$R, DCOCO, and Nucleus Speculative Decoding aim to improve efficiency by optimizing the verification process, restoring contextual dependencies, or allowing for more flexible acceptance rules. Some methods also explore analyzing LLM hidden states to better select layers for verification, while others focus on defending against attacks that degrade speculative decoding performance. AI

IMPACT These advancements in speculative decoding could significantly reduce LLM inference costs and latency, enabling wider deployment and more responsive AI applications.

RANK_REASON Multiple research papers published on arXiv detailing novel methods for speculative decoding in LLMs.

Read on arXiv cs.CL →

AI-generated summary · Google Gemini · from 22 sources. How we write summaries →

New methods accelerate LLM inference with speculative decoding · 10 sources tracked

How we ranked this

Signal score
0 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Research
Multiple research papers published on arXiv detailing novel methods for speculative decoding in LLMs.
Source corroboration
22 independent sources
Strong cross-source corroboration — multiple independent publishers covered this within the clustering window.
Topics
infra, paper
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
47 days old
Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.
Coverage growth since scoring
+3 source(s) since last score
New sources have picked up this story since our last re-score. Score will update on the next scoring pass.

Full methodology in our editorial standards.

COVERAGE [22]

  1. arXiv cs.CL TIER_1 English(EN) · Hanbin Bae, Heejin Choi, Joohyung Lee, Byoung Jin Choi, Sangjun Park, Hoon-Young Cho ·

    HyCo: Hybrid Drafting for Cost-Efficient Mobile-Cloud Speculative Decoding

    arXiv:2610.21436v1 Announce Type: new Abstract: Device-cloud speculative decoding reduces server computation, but growing a deep draft tree with a small language model (SLM) requires repeated passes on a mobile neural processing unit (NPU). We present HyCo, which assigns depth to…

  2. arXiv cs.CL TIER_1 English(EN) · Jihao Zhao, Qianyu Jiang, Shuaishuai Zu, Pengfei Li, Guanghuan Fang, Ben Feng, Biao Qin, Zhibin Wang, Baochuan Yang ·

    TreeSpark: Learning to Draft the Trees You Verify for Speculative Decoding

    arXiv:2610.20175v1 Announce Type: new Abstract: Parallel drafters such as DFlash and DSpark reduce proposal latency, but early mismatches limit how much of each block can be accepted. Retaining alternatives introduces parent states and candidate choices that reference-conditioned…

  3. arXiv cs.CL TIER_1 English(EN) · Shuxuan Li, Jialiang Kang, Yanghao Zhou, Ziye Zhu, Xiao Yang ·

    FOLD: Position-Free Far Memory with Local Order for Long-Context Speculative Decoding

    arXiv:2610.20392v1 Announce Type: new Abstract: Speculative decoding accelerates LLM generation through lightweight drafting and parallel verification. It is particularly promising for long inputs, where KV-cache access costs can be amortized over multiple accepted tokens. Yet mo…

  4. arXiv cs.AI TIER_1 English(EN) · Naji Alazhar, Wassel Alazhar ·

    Adaptive DHiSparse: Speculative Decoding over a Load-Adaptive KV Hierarchy

    arXiv:2610.13708v1 Announce Type: new Abstract: Long-context serving runs out of KV memory before the hardware reaches its useful throughput. HiSparse stores history in host memory and fetches only the selected entries, but it uses one cache budget at every load and does not hand…

  5. arXiv cs.AI TIER_1 English(EN) · Raghavv Goel, Sudhanshu Agrawal, Mukul Gagrani, Risheek Garrepalli, Mingu Lee ·

    Surfing the Activation Wave: Analyzing the Information and Geometry of LLM Hidden States for Speculative Decoding

    arXiv:2610.12939v1 Announce Type: cross Abstract: Decoder hidden states from different layers of large language models (LLMs) often encode complementary information. However, existing speculative decoding methods typically rely on a small set of target-model layers chosen using f…

  6. arXiv cs.CL TIER_1 English(EN) · Yuhang He, Hongyu Ge, Zihan Lin, Siyi Liu, Yongqi Zhang ·

    S$^3$R: Fast and Faithful RL Rollouts via Self-Sparse Speculative Decoding

    arXiv:2610.13202v1 Announce Type: cross Abstract: Fast rollout generation for long-context reinforcement learning with verifiable rewards (RLVR) requires reducing KV-cache traffic without changing the response groups used for policy optimization. In our profiled RLVR workloads, a…

  7. arXiv cs.CL TIER_1 English(EN) · Roy Eisenstadt, Harel Lidar, Tal Gonen, Omer Luxembourg, Itamar Zimerman, Lior Wolf, Eli David, Ilan Naiman ·

    Contextual Correction of Parallel Draftsfor Speculative Decoding

    arXiv:2610.12570v1 Announce Type: new Abstract: Speculative decoding accelerates large language model inference by using a lightweight drafter to propose multiple tokens that are subsequently verified in parallel by the target model. Recent diffusion-based drafters make this para…

  8. arXiv cs.AI TIER_1 English(EN) · Shijing Hu, Xuancheng Ren, Zhihui Lu, Pan Zhou ·

    Where Draft Trees Lose Target Mass: Exit-Guided Speculative Decoding

    arXiv:2610.11750v1 Announce Type: new Abstract: Tree-based speculative decoding verifies multiple draft continuations in one target-model pass, but finite trees built from draft scores face a fundamental draft-target mismatch. We ask whether better exact verification can increase…

  9. arXiv cs.AI TIER_1 English(EN) · Adam Y. J. Jones, Yu Yuan, Sergio Maffeis ·

    Speedbumps: Rejection Attacks on Speculative Decoding

    arXiv:2610.10929v1 Announce Type: cross Abstract: Speculative decoding is a popular technique for increasing the speed and reducing the costs of large language model (LLM) inference by verifying multiple draft tokens in a single target-model forward pass. The resulting benefit de…

  10. Hugging Face Daily Papers TIER_1 English(EN) ·

    Surfing the Activation Wave: Analyzing the Information and Geometry of LLM Hidden States for Speculative Decoding

    Decoder hidden states from different layers of large language models (LLMs) often encode complementary information. However, existing speculative decoding methods typically rely on a small set of target-model layers chosen using fixed, position-based heuristics. Selecting an opti…

  11. arXiv cs.CL TIER_1 English(EN) · Yunxiao Zhao, Changxiao Cai ·

    Training Parallel Speculative Draft Models by Directly Minimizing Expected Decoding Rounds

    arXiv:2610.10411v1 Announce Type: cross Abstract: Speculative decoding accelerates large language model inference by using a low-cost draft model to propose tokens that the full-size target model verifies in parallel. Parallel and semi-autoregressive (semi- AR) drafters improve d…

  12. Hugging Face Daily Papers TIER_1 English(EN) ·

    Contextual Correction of Parallel Draftsfor Speculative Decoding

    Speculative decoding accelerates large language model inference by using a lightweight drafter to propose multiple tokens that are subsequently verified in parallel by the target model. Recent diffusion-based drafters make this paradigm particularly attractive: deep attention-bas…

  13. arXiv cs.AI TIER_1 English(EN) · Chung-En Ho, Weiyu Sun, Cheng-Jhih Shih, He Li, Yong Liu, Yingyan Celine Lin ·

    SpecFold: Folding Multi-Branch Redundancy for Faster Speculative Decoding in Diffusion Language Models

    arXiv:2610.04875v2 Announce Type: replace Abstract: Diffusion large language models (DLLMs) generate text through iterative block denoising, and multi-branch speculative decoding accelerates this process by verifying a main branch together with multiple draft branches in a single…

  14. arXiv cs.AI TIER_1 English(EN) · Yichi Zhang, Zhiqi Wang, Neil Gong, Yuchen Yang ·

    Secure Speculative Decoding for Large Language Models

    arXiv:2610.08678v1 Announce Type: cross Abstract: Speculative decoding accelerates inference for a large language model (LLM), referred to as the \emph{target model}, by first using a smaller model, referred to as the \emph{draft model}, to generate candidate tokens and then veri…

  15. arXiv cs.CL TIER_1 English(EN) · Geonmo Gu, Byeongho Heo, HeeJae Jun, Yoohoon Kang, Sangmin Lee, Sangdoo Yun, Dongyoon Han ·

    DLoop: Looped Speculative Decoding

    arXiv:2610.07659v1 Announce Type: new Abstract: Speculative decoding accelerates autoregressive generation in large language models. In each drafting stage, a lightweight draft model proposes tokens that the target model subsequently verifies. With increasingly capable draft mode…

  16. arXiv cs.CL TIER_1 English(EN) · Sumin Lee, Sukmin Cho, Seungjae Lim, Youngjin Kwon ·

    AgSpec: Pushing the Limits of Retrieval-Based Speculative Decoding in Coding Agent Pipelines

    arXiv:2610.01108v2 Announce Type: replace Abstract: Retrieval-based speculative decoding (SD) drafts tokens by copying continuations from existing text, which suits coding agents that repeatedly reproduce code, logs, and earlier attempts. Yet existing methods fall short in agent …

  17. arXiv cs.CL TIER_1 English(EN) · Shuhao Li, Fanghua Ye, Wanyu Lin, Tianyu Yuan, Xiaoyu Shen ·

    Nucleus Speculative Decoding: Plausibility-Aware Verification Beyond Exact Distribution

    arXiv:2610.07822v1 Announce Type: new Abstract: Speculative decoding accelerates autoregressive generation by using a lightweight draft model to propose multiple tokens that are verified by a target model in parallel. However, the standard acceptance rule focuses on exact distrib…

  18. Hugging Face Daily Papers TIER_1 English(EN) ·

    SpecFold: Folding Multi-Branch Redundancy for Faster Speculative Decoding in Diffusion Language Models

    Diffusion large language models (DLLMs) generate text through iterative block denoising, and multi-branch speculative decoding accelerates this process by verifying a main branch together with multiple draft branches in a single forward pass. While prior DLLM acceleration methods…

  19. Hugging Face Daily Papers TIER_1 English(EN) ·

    DLoop: Looped Speculative Decoding

    Speculative decoding accelerates autoregressive generation in large language models. In each drafting stage, a lightweight draft model proposes tokens that the target model subsequently verifies. With increasingly capable draft models, we find that the target model frequently acc…

  20. r/LocalLLaMA TIER_1 English(EN) · /u/pmttyji ·

    [Paper] DLoop: Looped Speculative Decoding

    <table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1x1eu7o/paper_dloop_looped_speculative_decoding/"> <img alt="[Paper] DLoop: Looped Speculative Decoding" src="https://preview.redd.it/qt7fjeyd7euh1.png?width=140&amp;height=72&amp;auto=webp&amp;s=28336cf2022f4…

  21. dev.to — LLM tag TIER_1 English(EN) · jidonglab ·

    Speculative Decoding Made My vLLM Server Slower: The Acceptance Math

    <p>I added a draft model to my vLLM server, sent one curl request, and watched the tokens fly. Then I ran the load test with 64 concurrent users and total throughput went <em>down</em>. Same GPU, same target model, same prompts. The only change was one flag that every blog post s…

  22. r/LocalLLaMA TIER_1 English(EN) · /u/pmttyji ·

    [Paper] WaveFront Decoding: Parallelized Self-Speculative Decoding for Looped Language Models

    <table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1wz2e0o/paper_wavefront_decoding_parallelized/"> <img alt="[Paper] WaveFront Decoding: Parallelized Self-Speculative Decoding for Looped Language Models" src="https://preview.redd.it/3agb0qbdkuth1.png?width=14…