PulseAugur
中
实时 15:10:42
English(EN) Online Draft Co-Training for Speculative Decoding in Large-Scale, Long-Context RL Post-Training

新方法通过推测性解码提升大语言模型推理速度

作者 PulseAugur 编辑部 · [6 个来源] ·

研究人员正在开发用于加速大语言模型（LLM）推理的先进推测性解码技术。一种名为 X-CoSD 的方法，通过处理词汇差异和最小化数据传输，专注于设备上小型模型与大型服务器模型之间的有效通信。另一种名为 Osprey 的方法，利用预训练的小型语言模型作为草稿生成器，通过轻量级过程将其适配到各种目标模型，以提高接受率和令牌生成速度。此外，正在探索一种名为在线草稿协同训练（Online Draft Co-Training）的技术，通过优化注意力机制和阶段间特征传输，来提高强化学习后训练中草稿生成器的准确性，尤其适用于具有长上下文的大型模型。 AI

影响 这些推测性解码方面的进展旨在显著降低大语言模型推理的延迟和计算成本，从而可能实现更具响应性和效率的AI应用。

排序理由 多篇研究论文介绍了用于大语言模型推理中推测性解码的新技术。

在 arXiv cs.LG 阅读 →

AI 生成摘要 · Google Gemini · 来自 6 个来源。 我们如何撰写摘要 →

新方法通过推测性解码提升大语言模型推理速度

本文如何被排名

Signal score
0 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Research
多篇研究论文介绍了用于大语言模型推理中推测性解码的新技术。
Source corroboration
6 independent sources
Strong cross-source corroboration — multiple independent publishers covered this within the clustering window.
Topics
model release, infra
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
5 days old
Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.
Coverage growth since scoring
+1 source(s) since last score
New sources have picked up this story since our last re-score. Score will update on the next scoring pass.

完整方法见我们的编辑标准。

报道来源 [6]

  1. arXiv cs.CL TIER_1 English(EN) · Jaeduk Lee, Wan Choi ·

    X-CoSD：通信高效的跨词汇协作推测解码

    arXiv:2609.09166v1 Announce Type: new Abstract: This paper investigates collaborative speculative decoding (CoSD), a distributed large language model (LLM) inference framework in which an on-device small language model (SLM) drafts candidate tokens and a server LLM verifies them.…

  2. arXiv cs.CL TIER_1 English(EN) · Fengxiang Bie, Yuqing Jian, Yifan Yu, Zhongzhu Zhou, Zelei Shao, Ben Athiwaratkun, Shuaiwen Leon Song, Chenfeng Xu, Xiaoxia Wu, Tianyi Zhang ·

    Osprey：目标无关的预训练使投机性解码中的草稿生成器更强大

    arXiv:2609.09338v1 Announce Type: new Abstract: Speculative decoding is critical for accelerating LLM inference. However, the speedup is fragile: drafters are typically trained against a narrow distribution for a single target model, and their acceptance rate collapses under work…

  3. arXiv cs.LG TIER_1 English(EN) · Zili Wang, Zhaopeng Qiu, Yuekai Zhang, Shuang Yu, Junjie Lai ·

    面向大规模长上下文强化学习后训练的投机解码的在线草稿协同训练

    arXiv:2609.07108v1 Announce Type: new Abstract: Speculative decoding accelerates rollout generation, which dominates the cost of reinforcement learning (RL) post-training. Online co-training can further increase the draft's accuracy, yielding greater speedups. However, scaling th…

  4. Hugging Face Daily Papers TIER_1 English(EN) ·

    用于大规模、长上下文强化学习后期训练的投机解码的在线草稿协同训练

    A system for large-scale online draft co-training accelerates speculative decoding in RL post-training by extending context-parallel attention and adding cross-stage feature transport.

  5. dev.to — LLM tag TIER_1 English(EN) · oooocean66 ·

    MTP实践：在真实GPU上对Gemma的推测解码进行基准测试

    <p>In the concept edition, we saw that MTP (Multi-Token Prediction) lets a model predict several tokens ahead to speed up generation, and that Qwen and Gemma implement this in completely different ways.</p> <p>The theory makes sense, but how much faster does this actually make th…

  6. dev.to — LLM tag TIER_1 English(EN) · Kristiyan Stoyanov ·

    我如何为推测性解码训练了一个 DFlash Drafter

    <h1> If you prefer video format </h1> <p> </p> <h1> How I Trained a DFlash Drafter for Speculative Decoding </h1> <p>Running a capable local LLM is often easy. Making it responsive enough for interactive use, coding, or agent workflows is much harder.</p> <p>I wanted to improve t…

相关实体

相关话题