arXiv:2611.08669v1 Announce Type: cross Abstract: Speculative decoding has become a standard way to accelerate LLM inference: a lightweight drafter proposes tokens that the target verifies in one parallel pass. Existing drafters face trade-offs that limits their speedups: autoreg…
arXiv:2611.07263v1 Announce Type: new Abstract: Parallel block drafting accelerates speculative decoding by predicting multiple future positions in a single backbone pass, but on-device deployment constrains both drafter capacity and context storage. Under tight memory budgets, d…
arXiv:2610.31338v2 Announce Type: replace Abstract: Tree-based speculative decoding accelerates large language model inference by verifying multiple candidate branches in parallel. However, in practical serving scenarios, similar requests repeatedly induce overlapping candidate b…
arXiv:2611.07845v1 Announce Type: cross Abstract: Masked diffusion models generate text by reversing a stochastic masking process. Unlike autoregressive LLMs, which generate text one token at a time, they can reveal many tokens in parallel, at the price of an error, since tokens …
arXiv:2611.05611v1 Announce Type: new Abstract: Speculative decoding (SD) speeds up large language model (LLM) serving, but its gain depends on the workload's \emph{shape}, such as batch size and context length, and on its \emph{content}. A model-based drafter proposes a short bl…
arXiv:2611.05894v1 Announce Type: new Abstract: Recycling a target model's own predictions offers a training-free source of proposals for speculative decoding. In vision-language models, however, existing recycling indexes predictions by the last token alone, conflating predictio…
arXiv:2611.06425v1 Announce Type: new Abstract: Target-conditioned diffusion drafters predict candidate tokens in parallel, yet each new block must wait for updated target-model hidden states. This forces drafting and verification to run sequentially, limiting speedup even though…
arXiv cs.LG
TIER_1English(EN)·Hoang Anh Duy Le, Huibing Dong, Alireza Khadem, Ziteng Sun, Xiang Song, Yizheng Huang, Soroush Radpour, Anshumali Shrivastava, Ye He·
arXiv:2611.05724v1 Announce Type: new Abstract: Speculative decoding accelerates large language model inference with a small drafter, whose per-guess acceptance rate determines the achievable speedup. Feature-level drafters share a design choice: every token, easy or hard, is dra…
Parallel block drafting accelerates speculative decoding by predicting multiple future positions in a single backbone pass, but on-device deployment constrains both drafter capacity and context storage. Under tight memory budgets, drafters must model long-range dependencies and f…
D3D is a standalone diffusion drafter that removes target-model hidden-state dependencies to enable asynchronous speculative decoding with high speedup.
arXiv:2610.41177v1 Announce Type: new Abstract: LLM agents execute tasks through repeated model calls over evolving histories of reasoning, actions, and tool feedback. Aggregate speculative-decoding metrics merge these distinct decisions, providing limited guidance on where draft…
SpecOPD accelerates speculative decoding by parallelizing on-policy draft supervision across independent branches, improving acceptance length and training throughput across diverse model architectures and tasks.
SpecPilot autonomously discovers deployment-specific algorithms and system optimizations to maximize speculative decoding speedups across diverse models, drafters, and serving regimes without quality loss.
arXiv cs.AI
TIER_1English(EN)·Abhishek Malvankar, Paul Castro, Nick Mitchell, Lionel Villard·
arXiv:2610.38968v1 Announce Type: cross Abstract: Speculative decoding speeds up large language model (LLM) serving only while the GPU has spare compute, so the best draft length, the number of tokens a small drafter guesses per step, shrinks as the batch grows. vLLM lets operato…
arXiv:2610.38981v1 Announce Type: cross Abstract: Speculative decoding (SD) accelerates autoregressive inference by leveraging the memory bound nature of decoding, verifying multiple draft tokens in a single forward pass. For mixture-of-experts (MoE) models, prior work predicts a…
arXiv cs.AI
TIER_1English(EN)·Bole Ma, Ayesha Afzal, Jan Eitzinger, Harald Koestler, Gerhard Wellein·
arXiv:2610.40287v1 Announce Type: cross Abstract: Large language model serving is limited by facility power. Energy per token determines how many tokens a datacenter can serve, while a latency target bounds how quickly each user receives them. During decode, every weight is read …
arXiv cs.CV
TIER_1English(EN)·Hong Nie, Zhangchi Gao, Yuqian Liang, Jun Yu, Bo Wang, Jianqing Sun, Shengping Liu·
arXiv:2611.01617v2 Announce Type: replace-cross Abstract: Speculative decoding accelerates vision-language model (VLM) inference without altering the output distribution, yet two factors limit its gains. Existing drafters either lack the image evidence required for visually groun…
<h2> TL;DR </h2> <p>Tencent가 11월 7일 HF에 올린 <code>Qwen3.8-27B-DSpine-b16</code>은 Qwen3.8-27B의 속도를 높이기 위한 <strong>투기 디코딩 초안 모델</strong>입니다. 초안 모델이 한 번에 16토큰을 병렬로 제안하고, 큰 모델이 그것을 한꺼번에 검증하는 방식입니다. 카드에는 비교 대상인 DFlash2보다 수락 길이가 길다는 표와, 자기회귀 디코딩 대비 최대 3.84배라는 처리량 표가 있습니다.</p> <p>저희는 이 체…
