English(EN) Geometry-Aware Online Scheduling for LLM Serving: From Theoretical Bound to System Practice

新的SVF算法通过考虑几何内存增长来优化LLM服务

作者 PulseAugur 编辑部 · [1 个来源] · 2026-06-21 04:05

研究人员开发了一种新的几何感知在线调度算法，称为最小体积优先（SVF）及其高效变体1位SVF，以优化大型语言模型（LLM）服务。该方法通过考虑LLM推理动态的二维时空几何增长，解决了传统以时间为中心的调度启发式方法的局限性。理论分析表明SVF提高了竞争比，并将其集成到vLLM与Llama-3.1模型中的实际应用证明了延迟的显著降低和竞争性吞吐量的提升。 AI

影响这种新的调度方法可以显著提高大型语言模型的服务效率并降低成本。

排序理由详细介绍新算法及其理论和实践评估的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.AI 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 0 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 详细介绍新算法及其理论和实践评估的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: paper, infra; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: 112 days old; Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.

完整方法见我们的编辑标准。

报道来源 [1]

arXiv cs.AI TIER_1 English(EN) · Zijie Zhou · 2026-06-21 04:05

面向LLM服务的几何感知在线调度：从理论界限到系统实践

The explosive demand for interactive Large Language Model serving has highlighted the management of the Key-Value cache's dynamic memory footprint as a critical area for performance optimization in inference engines. Modern inference systems overwhelmingly rely on time-centric sc…

本文如何被排名

报道来源 [1]

面向LLM服务的几何感知在线调度：从理论界限到系统实践

相关实体

相关话题