一位用户显著优化了 Qwen3.8-27B 模型在 RTX 3090 GPU 上的推理性能，实现了大幅的性能提升。通过一系列优化，包括 FP8 KV 缓存、模型头部的 INT8 量化以及 DFlash2 和查找增强等先进的草稿技术，用户将单请求吞吐量推至 138 tps，并发吞吐量推至 900 tps 以上。这些改进还极大地降低了对话中后续轮次的延迟，并将上下文长度支持到 262k tokens，同时保持了模型质量。 AI
影响 展示了在消费级硬件上优化现有开源 LLM 的巨大潜力，提高了可访问性和性能。
排序理由 用户驱动的针对特定硬件的现有开源模型优化，而非来自前沿实验室的新模型发布。
AI 生成摘要 · Google Gemini · 来自 5 个来源。 我们如何撰写摘要 →
<!-- SC_OFF --><div class="md"><p>Four days ago I released a <a href="https://www.reddit.com/r/LocalLLaMA/comments/1vq6fdj/qwen3827b_on_rtx_3090_82_tps_single_request_up_to/">hyper-optimized Qwen3.8-27B inference engine</a> for an RTX 3090 (82 tps single request, 672 peak). Since…
<!-- SC_OFF --><div class="md"><p>Edit: Title says 134 tps, it's actually 138 -- keep in mind my 3090 is power limited to 250w. </p> <p>Three days ago I released a hyper-optimized Qwen3.8-27B inference engine for an RTX 3090 (82 tps single request, 672 peak), and <a href="https:/…
<!-- SC_OFF --><div class="md"><p>Two days ago I released a <a href="https://www.reddit.com/r/LocalLLaMA/comments/1vq6fdj">hyper-optimized Qwen3.8-27B</a> inference engine for an RTX 3090 (82 tps single request, 672 peak) - <a href="https://www.reddit.com/r/LocalLLaMA/comments/1v…
<!-- SC_OFF --><div class="md"><p>I'm back.</p> <p>Yesterday I released <a href="https://www.reddit.com/r/LocalLLaMA/comments/1vq6fdj/comment/p46v06y/?screen_view_count=2&ext-referrer=DIRECT&sort=new">the first version </a>of hyper-optimized Qwen3.8-27B inference engine f…
<!-- SC_OFF --><div class="md"><p>Hi,</p> <p>After a long night of optimizations, I believe I have made the fastest inference engine for Qwen3.6-28B on a 3090.</p> <p>Quick metrics:</p> <p>- 250w power capped</p> <p>- Up to 195k context (ships with 150k for safety though)</p> <p>…