研究人员开发了 Galahad，这是一种新颖的内存层，旨在使大型语言模型 (LLM) 读取成为一次性成本，从而显著提高 LLM 服务效率。Galahad 通过保存和加载模型的键值 (KV) 状态来解决无状态服务的问题，防止对已处理文本进行重复计算。这项创新将 LLM 服务从无状态推理模型转变为有状态推理模型，在召回测试中速度和能耗方面得到了显著改善。 AI
影响 这一发展可以显著降低 LLM 服务的计算成本和延迟，使其在实时应用中更有效率。
排序理由 该集群描述了一篇关于 LLM 新颖内存层的新研究论文。
在 Hugging Face Daily Papers 阅读 →
AI 生成摘要 · Google Gemini · 来自 4 个来源。 我们如何撰写摘要 →
完整方法见我们的编辑标准。
arXiv:2609.39358v1 Announce Type: cross Abstract: A transformer language model performs a bounded amount of computation per token, and recent work by Vishal Sikka, former CEO of Infosys, argues that this bound limits which tasks a model can carry out or verify (arXiv:2507.07505).…
A transformer language model performs a bounded amount of computation per token, and recent work by Vishal Sikka, former CEO of Infosys, argues that this bound limits which tasks a model can carry out or verify (arXiv:2507.07505). We ask how much of the budget beneath that ceilin…
A transformer language model performs a bounded amount of computation per token, and recent work by Vishal Sikka, former CEO of Infosys, argues that this bound limits which tasks a model can carry out or verify (arXiv:2507.07505). We ask how much of the budget beneath that ceilin…
A transformer language model performs a bounded amount of computation per token, and recent work by Vishal Sikka, former CEO of Infosys, argues that this bound limits which tasks a model can carry out or verify (arXiv:2507.07505). We ask how much of the budget beneath that ceilin…