PulseAugur
中
实时 20:23:38
English(EN) How Lossless Is Lossless Speculative Decoding? The Role of Numerical Precision in Orthrus

新的投机式解码方法提高了LLM推理速度 · 跟踪7个来源

研究人员正在推进大型语言模型的投机式解码技术,以提高推理速度。两篇新的arXiv论文,ECHO和LoopSpec,分别引入了分层和流水线方法来优化草稿候选生成和验证。ECHO使用早期层奖励logits进行快速草稿探索,而LoopSpec则利用循环Transformer中的中间状态进行流水线草稿生成。另一篇关于Orthrus的论文质疑了投机式解码的“无损”声明,强调了数值精度对输出一致性的影响,而另一项关于Carryover Drafting的研究提出回收被拒绝的状态以增强接受长度和整体加速。 AI

影响 这些在投机式解码方面的进展可以显著降低LLM的推理延迟和计算成本,从而实现更广泛的部署和更高效的大模型使用。

排序理由 多篇学术论文发表在arXiv上,详细介绍了LLM投机式解码技术的新方法和分析。

在 Hugging Face Daily Papers 阅读 →

AI 生成摘要 · Google Gemini · 来自 11 个来源。 我们如何撰写摘要 →

新的投机式解码方法提高了LLM推理速度 · 跟踪7个来源

本文如何被排名

Signal score
0 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Research
多篇学术论文发表在arXiv上,详细介绍了LLM投机式解码技术的新方法和分析。
Source corroboration
11 independent sources
Strong cross-source corroboration — multiple independent publishers covered this within the clustering window.
Topics
paper, infra
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
69 days old
Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.
Coverage growth since scoring
+3 source(s) since last score
New sources have picked up this story since our last re-score. Score will update on the next scoring pass.

完整方法见我们的编辑标准。

报道来源 [11]

  1. arXiv cs.CL TIER_1 English(EN) · Roy Eisenstadt, Ido Cohen, Edo Cohen-Karlik, Lior Wolf, Itamar Zimerman ·

    复制还是不复制:通过内在模型信号控制推测性解码

    arXiv:2609.20186v1 Announce Type: new Abstract: Speculative Decoding (SD) has significantly accelerated Large Language Model (LLM) inference, yet existing approaches face a fundamental tradeoff between two drafting strategies: neural drafting and context-based copying. Neural dra…

  2. arXiv cs.CL TIER_1 English(EN) · Amir Ziashahabi, Hossein Entezari Zarch, Lei Gao, Murali Annavaram, Salman Avestimehr ·

    ASPIRE:长上下文大语言模型推理的异步批处理自推测解码

    arXiv:2609.17943v1 Announce Type: cross Abstract: Long-context LLM inference is bottlenecked by attention, whose repeated KV-cache reads make decoding memory-bound. Self-speculative decoding alleviates this by drafting tokens with sparse attention and verifying them with full att…

  3. arXiv cs.CL TIER_1 English(EN) · SangLyul Cho, Langqing Cui, Sehoon Kim, Dongsu Han, Insu Han ·

    LoopSpec:用于循环 Transformer 的流水线自推测解码

    arXiv:2609.17184v1 Announce Type: cross Abstract: Looped Transformers achieve strong performance with compact parameter sizes by repeatedly applying a shared stack of Transformer blocks across recurrent depths. However, they incur higher decoding latency than standard Transformer…

  4. arXiv cs.CL TIER_1 English(EN) · Ziyang Ma, Zihong Zhang, Zuchao Li, Lefei Zhang, Baoyuan Qi, Siqi Li, Simin Yu ·

    ECHO:投机解码中的早期层协作分层编排及奖励对数

    arXiv:2609.17241v1 Announce Type: new Abstract: While draft-model-free speculative decoding offers a promising path to efficient LLM inference, it is frequently constrained by stale draft candidates and the high computational cost of the verification. To address these challenges,…

  5. Hugging Face Daily Papers TIER_1 English(EN) ·

    ECHO:在投机解码中通过奖励对数实现早期层协作分层编排

    While draft-model-free speculative decoding offers a promising path to efficient LLM inference, it is frequently constrained by stale draft candidates and the high computational cost of the verification. To address these challenges, we propose ECHO, a hierarchical dual-loop frame…

  6. arXiv cs.AI TIER_1 English(EN) · Jahyun Koo, Sunghyeon Woo, Jaeeun Kil, Jeongtae Lee, Sungjae Lee, Kyomin Jung, Minsub Kim ·

    遗留草稿:利用被拒绝的状态进行推测性解码

    arXiv:2609.14717v1 Announce Type: cross Abstract: Speculative decoding accelerates LLM inference by verifying multiple drafted tokens in parallel, allowing a single target forward pass to accept several tokens. By construction, verification computes representations for both accep…

  7. arXiv cs.AI TIER_1 English(EN) · Ilya Koziev, Leonid Sinev, Ivan Oseledets ·

    无损投机解码有多无损?数值精度在Orthrus中的作用

    arXiv:2609.15504v1 Announce Type: cross Abstract: Orthrus is a hybrid autoregressive-diffusion architecture that accelerates autoregressive language-model inference by generating multiple tokens in parallel while using a frozen autoregressive backbone. Its central claim is that a…

  8. Hugging Face Daily Papers TIER_1 English(EN) ·

    无损投机解码有多无损?数值精度在Orthrus中的作用

    Orthrus is a hybrid autoregressive-diffusion architecture that accelerates autoregressive language-model inference by generating multiple tokens in parallel while using a frozen autoregressive backbone. Its central claim is that an intra-model consensus mechanism enables lossless…

  9. Hugging Face Daily Papers TIER_1 English(EN) ·

    无损投机解码有多无损?数值精度在Orthrus中的作用

    Orthrus achieves lossless speculative decoding only under high numerical precision, with trajectory divergence under BF16 that does not impair downstream benchmarks.

  10. dev.to — LLM tag TIER_1 English(EN) · Manoranjan Rajguru ·

    2026年推测性解码:从EAGLE到DFlash再到XPress——工程师的完整手册

    <h1> Speculative Decoding in 2026: From EAGLE to DFlash to XPress — The Complete Engineer's Playbook </h1> <p><a class="article-body-image-wrapper" href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-upl…

  11. Mastodon — mastodon.social TIER_1 English(EN) · [email protected] ·

    2026年推测性解码:从EAGLE到DFlash再到XPress — 工程师的完整指南... # ai # llm # machinelearning # programming # software # coding # devel

    Speculative Decoding in 2026: From EAGLE to DFlash to XPress — The Complete Engineer's... # ai # llm # machinelearning # programming # software # coding # development # engineering # inclusive # community Speculative Decoding in 2026: From EAGLE to DFlash to XPress — The Complete…