English(EN)Demystifying Speculative Decoding: From Architecture to Production Bottlenecks
新研究提升LLM推测性解码的速度和准确性
作者PulseAugur 编辑部·[18 个来源]·
多篇研究论文正在探索大型语言模型（LLMs）推测性解码的进展，旨在提高推理速度和输出质量。一种名为SpecPV的方法通过对关键值状态进行部分验证，在长上下文生成任务上实现了高达6倍的加速。另一种方法，熵感知令牌拒绝（EASD），通过检测和拒绝不确定的令牌预测来提高推理质量，有时甚至超越目标模型本身的性能。其他研究则解决了批处理推测性解码中的锯齿张量问题，并引入了感知验证的训练技术来优化草稿模型。
AI
arXiv:2609.02897v1 Announce Type: new Abstract: Speculative decoding accelerates LLM inference by drafting candidate tokens and verifying them in parallel. Tree-attention drafters such as EAGLE-3 are widely adopted, yet typically hold two decisions fixed: (1) a strict token-match…
arXiv cs.CL
TIER_1English(EN)·Ximing Dong, Shaowei Wang, Dayi Lin, Boyuan Chen, Ahmed E. Hassan·
arXiv:2602.03708v3 Announce Type: replace Abstract: Large Language Models (LLMs) achieve strong performance across many tasks but suffer from high inference latency due to autoregressive decoding. The issue is exacerbated in Large Reasoning Models (LRMs), which generate lengthy c…
arXiv cs.CL
TIER_1English(EN)·Weihuang Wen, Yingying Liu, Yichuan Liu, Wenqi Zeng, Li Zhou, Chumin Sun, Jie Sun, Tianshu Yu·
arXiv:2609.01068v1 Announce Type: new Abstract: The heavy-tailed distribution of output lengths in Large Language Model (LLM) serving poses major challenges for resource provisioning and cluster scheduling. Although output-length prediction can mitigate these issues, existing app…
arXiv cs.CL
TIER_1English(EN)·Guanqiao Chen, Di Wang, Lijie Hu·
arXiv:2609.00796v1 Announce Type: new Abstract: As one of the most critical challenges in large language models, contextual faithfulness directly determines their reliability in knowledge-intensive applications. This task is particularly challenging as it requires balancing factu…
arXiv cs.AI
TIER_1English(EN)·Jungseob Lee, Seongtae Hong, Dongyub Jude Lee, Chanjun Park, Jaehyung Seo, Sugyeong Eo, Heuiseok Lim·
arXiv:2609.00355v1 Announce Type: new Abstract: Speculative decoding accelerates generation without changing its output, yet on vision-language models (VLMs) it has been caught in a self-defeating cycle. The drafter stays autoregressive, so it must stay small. A small drafter can…
arXiv:2608.30252v1 Announce Type: new Abstract: Long-context LLM applications such as document summarization and multi-turn agents require generation from prefixes spanning tens of thousands of tokens, making decoding latency a major bottleneck. Speculative decoding (SD) reduces …
arXiv:2512.02337v2 Announce Type: replace Abstract: Growing demands from tasks like code generation, deep reasoning, and long-document understanding have made long-context generation a crucial capability for large language models (LLMs). Speculative decoding is one of the most di…
arXiv:2510.22876v4 Announce Type: replace-cross Abstract: Inference optimizations are routinely evaluated by throughput alone, without verifying output correctness. We conduct a forensic analysis of batch speculative decoding and find that several widely-used implementations sile…
arXiv cs.AI
TIER_1English(EN)·Tiancheng Su, Meicong Zhang, Guoxiu He·
arXiv:2512.23765v2 Announce Type: replace-cross Abstract: Speculative decoding (SD) accelerates large language model (LLM) inference by using a lightweight draft model to propose tokens and a stronger target model to verify them. However, standard SD is mainly designed for accele…
arXiv:2608.30135v1 Announce Type: new Abstract: Speculative decoding accelerates large language model inference by using a draft model to generate candidate tokens, which are verified by the target model in a single forward pass. Verification proceeds sequentially and discards ev…
arXiv:2608.30427v1 Announce Type: new Abstract: Speculative decoding speeds up generation with an efficient draft model (drafter) that proposes tokens for a target model to verify in one pass, preserving the target's output distribution. High-acceptance block-diffusion drafters s…
Long-context LLM applications such as document summarization and multi-turn agents require generation from prefixes spanning tens of thousands of tokens, making decoding latency a major bottleneck. Speculative decoding (SD) reduces latency without changing model outputs, but its …
arXiv:2608.28099v1 Announce Type: cross Abstract: Real-time classification during language model inference is valuable for safety filtering, behavioral analysis, and model monitoring, but current approaches force a trade-off between accuracy and efficiency. Hidden-state probes ar…
Verification-Aware Training improves draft models for speculative decoding by simulating sequential verification during training and adapting loss weights to acceptance patterns.
arXiv cs.CL
TIER_1English(EN)·Jiaming Fan, Daming Cao, Canchen Huang, Jiale Fu, Jin Zhang, Junjie Gao, Kai Yang, Xiangzhong Luo, Xu Yang·
arXiv:2608.26112v1 Announce Type: new Abstract: Speculative decoding accelerates large language model inference through a draft-then-verify paradigm. Building on this, tree-structured methods improve inference by organizing proposals into multiple candidate paths, increasing the …
dev.to — LLM tag
TIER_1English(EN)·Yuvraj singh Bhadoria·
<h1> Demystifying Speculative Decoding: From Architecture to Production Bottlenecks </h1> <p>Speculative decoding is one of the most widely discussed inference optimizations in recent LLM engineering, and frequently one of the most misunderstood. The core proposition sounds ideal…
dev.to — LLM tag
TIER_1English(EN)·Yuvraj singh Bhadoria·
<h1> Demystifying Speculative Decoding: From Architecture to Production Bottlenecks </h1> <p>Speculative decoding is one of the most widely discussed inference optimizations in recent LLM engineering, and frequently one of the most misunderstood. The core proposition sounds ideal…
dev.to — LLM tag
TIER_1English(EN)·Yuvraj singh Bhadoria·
<h1> Demystifying Speculative Decoding: From Architecture to Production Bottlenecks </h1> <p>Speculative decoding is one of the most widely discussed inference optimizations in recent LLM engineering, and frequently one of the most misunderstood. The core proposition sounds ideal…
