English(EN) DualPace: Decoupled Scheduling and Cooperative GPU--CPU Decoding for LLM Serving

DualPace运行时通过GPU-CPU协同提升LLM服务吞吐量

作者 PulseAugur 编辑部 · [1 个来源] · 2026-10-28 04:00

研究人员开发了DualPace，一种旨在提高推理吞吐量和降低延迟的新型LLM服务运行时。DualPace将GPU和CPU引擎之间的调度解耦，允许CPU独立处理请求并更有效地利用主机内存。与vLLM和NEO等现有系统相比，这种协同方法可将服务吞吐量提高高达90.6%，同时保持可比的延迟。 AI

影响 DualPace的方法可以显著提高部署大型语言模型的效率和成本效益。

排序理由该集群描述了一篇关于LLM服务新颖系统的研究论文。[lever_c_demoted from research: ic=1 ai=1.0]

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 18 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 该集群描述了一篇关于LLM服务新颖系统的研究论文。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: infra, other; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: Breaking (< 6h); Fresh story with cross-source coverage still developing. Ranking may shift as more sources report.

完整方法见我们的编辑标准。

arXiv cs.LG TIER_1 English(EN) · YuXuan Lin, KeQiang Wu, Kingsum Chow · 2026-10-28 04:00

DualPace：LLM服务解耦调度与协同GPU--CPU解码

arXiv:2610.33501v1 Announce Type: cross Abstract: Large language model (LLM) serving relies on batching to improve inference throughput, but the key-value (KV) cache grows with sequence length and limits GPU-resident concurrency. CPU offloading can expand KV capacity using host m…