PulseAugur
中
实时 07:50:04
English(EN) JigSLO: Solving the Resource Puzzle for Efficient Multi-LLM Serving

JigSLO系统优化共享GPU上的多LLM服务

研究人员开发了JigSLO,一个旨在优化共享图形处理单元(GPU)上多个大型语言模型(LLM)服务的新颖系统。JigSLO通过将LLM的预填充(prefill)和解码(decode)阶段视为独立实例来解决内存带宽争用问题,从而实现互补的共址(co-location)。这种方法根据受争用和不受争用的时段估算延迟,使系统能够在满足延迟目标的同时最大化吞吐量。在聊天、代码生成和摘要任务的评估中,JigSLO展示了显著的改进,与现有方法相比,实现了2.3倍更好的SLO达成率和1.2倍更高的吞吐量,同时在长上下文工作负载中使用的GPU数量减半。 AI

影响 优化LLM服务效率,可能降低多LLM部署的基础设施成本并改善响应时间。

排序理由 该集群包含一篇详细介绍LLM服务新系统的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.LG 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →

JigSLO系统优化共享GPU上的多LLM服务

本文如何被排名

Signal score
27 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Tool
该集群包含一篇详细介绍LLM服务新系统的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration
Single-source cluster
Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics
infra, paper
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
Breaking (< 6h)
Fresh story with cross-source coverage still developing. Ranking may shift as more sources report.

完整方法见我们的编辑标准。

报道来源 [1]

  1. arXiv cs.LG TIER_1 English(EN) · Kyungmin Bin, Seungbeom Choi, Jimyoung Son, Jieun Choi, Daseul Bae, Daehyeon Baek, Kihyo Moon, Minsung Jang, Hyojung Lee ·

    JigSLO:为高效多LLM服务解决资源难题

    arXiv:2509.06261v3 Announce Type: replace-cross Abstract: GPU spatial sharing co-locates instances with complementary demands, such as compute-bound prefill and memory-bound decode, promising cost-efficient multi-LLM serving. Prior systems partition compute and coordinate shared …