New methods accelerate LLM inference with speculative decoding · 7 sources tracked
ByPulseAugur Editorial·[9 sources]·
Researchers are developing new methods to accelerate the inference speed of large language models (LLMs) through speculative decoding. DARTree and SPADE are two such approaches, with DARTree focusing on tree-based speculative decoding for improved acceptance length and speedup, while SPADE integrates speculative decoding across edge and cloud devices to reduce costs and latency. Other related work includes MemSpec, which optimizes adaptive speculative decoding for memory-constrained edge devices, and Goose, which uses anisotropic speculation trees to enhance efficiency. These advancements aim to make LLM deployment more practical and cost-effective.
AI
IMPACT
Accelerates LLM inference, potentially reducing deployment costs and latency for AI applications.
RANK_REASON
Multiple research papers introducing new methods for speculative decoding in LLMs.
arXiv:2608.14787v1 Announce Type: cross Abstract: Speculative decoding is a leading technique to reduce the cost of autoregressive generation by using a small drafter to propose several tokens, which are then verified in parallel by a larger target model. Speculative diffusion de…
arXiv:2608.13076v1 Announce Type: new Abstract: Large Language Models (LLMs) have achieved remarkable success in natural language understanding and generation, but their deployment is constrained by high computational demands. Deploying smaller LLMs directly on the edge can circu…
arXiv:2608.13524v1 Announce Type: new Abstract: Speculative decoding losslessly accelerates autoregressive language models by verifying multiple draft tokens in parallel. Diffusion-based drafters further reduce proposal latency by predicting an entire token block in parallel, but…
Speculative decoding losslessly accelerates autoregressive language models by verifying multiple draft tokens in parallel. Diffusion-based drafters further reduce proposal latency by predicting an entire token block in parallel, but their position-wise distributions are marginal …
arXiv:2511.18191v2 Announce Type: replace Abstract: Time series forecasting drives operational decisions under tight latency budgets, and autoregressive time series foundation models (TSFMs) increasingly deliver the most accurate forecasts. That accuracy is paid for at inference,…
arXiv cs.AI
TIER_1English(EN)·Eunjeong Kim, Yeong Jun Jeon, Myeonggyun Han·
arXiv:2608.10362v1 Announce Type: cross Abstract: Speculative decoding accelerates autoregressive large language model (LLM) inference by using a lightweight draft model to speculate multiple tokens, reducing expensive target model decoding steps. Its effectiveness depends heavil…
arXiv cs.AI
TIER_1English(EN)·Zexun Lin, Yuan Feng, Junlin Lv, Kevin S. Zhou, Xike Xie·
arXiv:2608.08721v1 Announce Type: cross Abstract: Speculative decoding accelerates large language model inference by drafting multiple tokens for parallel verification, with efficiency critically determined by the speculative length selected at each decoding round. Existing dynam…
arXiv:2604.02047v2 Announce Type: replace-cross Abstract: Speculative decoding accelerates large language model inference by drafting multiple candidate tokens and verifying them in a single forward pass. Candidates are organized as a tree: deeper trees accept more tokens per ste…
