UniVer和SpecKV等新技术通过推测性解码提升LLM推理速度

报道来源 [7]

1. arXiv cs.LG TIER_1 English(EN) · Yepeng Weng, Qiao Hu, Takehisa Yairi · 2026-05-07 04:00

   UniVer：多步多草稿推测性解码的统一视角

   arXiv:2605.04543v1 Announce Type: cross Abstract: Speculative decoding accelerates Large Language Models via draft-then-verify, where verification can be framed as an Optimal Transport (OT) problem. Existing approaches typically handle multi-draft and multi-step aspects in isolat…

2. arXiv cs.CL TIER_1 English(EN) · Takehisa Yairi · 2026-05-06 06:42

   UniVer：多步多草稿推测性解码的统一视角

   Speculative decoding accelerates Large Language Models via draft-then-verify, where verification can be framed as an Optimal Transport (OT) problem. Existing approaches typically handle multi-draft and multi-step aspects in isolation, applying either flat OT to single-step drafts…

3. arXiv cs.LG TIER_1 Română(RO) · Tanishq Kumar, Tri Dao, Avner May · 2026-05-06 04:00

   推测性推测解码

   arXiv:2603.03251v3 Announce Type: replace Abstract: Autoregressive decoding is bottlenecked by its sequential nature. Speculative decoding has become a standard way to accelerate inference by using a fast draft model to predict upcoming tokens from a slower target model, and then…

4. arXiv cs.CL TIER_1 English(EN) · Shikhar Shukla · 2026-05-05 04:00

   SpecKV：自适应推测解码与感知压缩的Gamma选择

   arXiv:2605.02888v1 Announce Type: cross Abstract: Speculative decoding accelerates large language model (LLM) inference by using a small draft model to propose candidate tokens that a larger target model verifies. A critical hyperparameter in this process is the speculation lengt…

5. arXiv cs.CL TIER_1 English(EN) · Shikhar Shukla · 2026-05-04 17:55

   SpecKV：自适应推测解码与感知压缩的Gamma选择

   Speculative decoding accelerates large language model (LLM) inference by using a small draft model to propose candidate tokens that a larger target model verifies. A critical hyperparameter in this process is the speculation length $γ$, which determines how many tokens the draft …

6. arXiv cs.CL TIER_1 English(EN) · Shikhar Shukla · 2026-05-04 17:55

   SpecKV：自适应推测解码与感知压缩的Gamma选择

   Speculative decoding accelerates large language model (LLM) inference by using a small draft model to propose candidate tokens that a larger target model verifies. A critical hyperparameter in this process is the speculation length~$γ$, which determines how many tokens the draft …

7. arXiv cs.LG TIER_1 English(EN) · Muhammad Shafique, Abdul Basit, Muhammad Abdullah Hanif, Alberto Marchisio, Rachmad Vidya Wicaksana Putra, Minghao Shao · 2026-04-27 04:00

   焦点会议：加速多模态基础模型的硬件与软件技术

   arXiv:2604.21952v1 Announce Type: new Abstract: This work presents a multi-layered methodology for efficiently accelerating multimodal foundation models (MFMs). It combines hardware and software co-design of transformer blocks with an optimization pipeline that reduces computatio…