English(EN) Toward Robust and Efficient ML-Based GPU Caching for Modern Inference

新的基于机器学习的GPU缓存算法LCR提升LLM推理速度

作者 PulseAugur 编辑部 · [1 个来源] · 2026-04-27 04:00

研究人员开发了一种名为学习增强LRU (LALRU) 的新GPU缓存算法，旨在提高AI推理期间的效率。该算法将学习到的预测与缓存策略相结合，以确保在预测准确时接近最优，并在预测不准确时限制性能下降。基于LALRU的一个实际实现LCR，在LLM工作负载中表现出显著的改进，将P99首个令牌时间缩短了高达28.3%，并将DLRM工作负载的吞吐量提高了高达24.2%。 AI

影响提高了LLM和DLRM工作负载的推理效率和吞吐量，可能降低运营成本。

排序理由学术论文，介绍了一种用于AI推理中GPU缓存的新算法。

在 arXiv cs.LG 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 0 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Research; 学术论文，介绍了一种用于AI推理中GPU缓存的新算法。
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: paper, infra; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: 156 days old; Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.

完整方法见我们的编辑标准。

报道来源 [1]

arXiv cs.LG TIER_1 English(EN) · Peng Chen, Jiaji Zhang, Hailiang Zhao, Yirong Zhang, Shenyao Chen, Jiahong Yu, Xueyan Tang, Yixuan Wang, Hao Li, Jianping Zou, Gang Xiong, Kingsum Chow, Shuibing He, Shuiguang Deng · 2026-04-27 04:00

面向现代推理的鲁棒高效的基于机器学习的GPU缓存

arXiv:2509.20979v2 Announce Type: replace Abstract: In modern GPU inference, cache efficiency remains a major bottleneck, and heuristic policies such as \textsc{LRU} can perform far worse than the offline optimum. Existing learning-based caching systems improve hit rates mainly t…

本文如何被排名

报道来源 [1]

面向现代推理的鲁棒高效的基于机器学习的GPU缓存

相关实体

相关话题