English(EN) Prefill vs Decode: Why Splitting Them Wins

vLLM 0.33 拆分 LLM 处理以消除用户提示冻结

作者 PulseAugur 编辑部 · [1 个来源] · 2026-11-13 17:06

vLLM 发布了 0.33 版本，引入了重大的架构变更，以解决大型语言模型的延迟问题。此次更新将 LLM 处理的“预填充”和“解码”阶段分开，并将它们分配到不同的 GPU 池。这种分离可以防止计算密集型的预填充阶段（处理用户提示）冻结内存带宽受限的解码阶段（负责生成 token）。 AI

影响此次更新旨在通过减少延迟和消除提示处理期间的冻结，显著改善 LLM 应用程序的用户体验。

排序理由这是 LLM 服务框架的软件更新，并非新的模型发布或核心研究。

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 24 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 这是 LLM 服务框架的软件更新，并非新的模型发布或核心研究。
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: infra, product; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: Breaking (< 6h); Fresh story with cross-source coverage still developing. Ranking may shift as more sources report.

完整方法见我们的编辑标准。

dev.to — LLM tag TIER_1 English(EN) · Daniel Sam Pete Thiyagu · 2026-11-13 17:06

预填充与解码：为何拆分它们能获胜

<p>A user pastes an 8,000-token document into a chat box and hits send. On the server, that prompt takes about 320 milliseconds of pure GPU math to process. For those 320 milliseconds, every other user already chatting on that same GPU gets nothing — their text freezes mid-senten…