English(EN) 4 coding agents on one RTX 3090: ~290 tok/s total from a custom CUDA engine for Qwen3.8-27B (3-4x llama-server --parallel)

OpenJet引擎使4个编码代理在同一块RTX 3090上以290 tok/s运行

作者 PulseAugur 编辑部 · [1 个来源] · 2026-10-27 13:15

一款名为OpenJet的新型定制CUDA引擎已被开发出来，用于在单个GPU（特别是RTX 3090）上同时运行多个编码代理。该引擎允许四个代理的总吞吐量约为每秒290个令牌，这比现有的解决方案（如llama-server，在相同配置下实现约每秒72-80个令牌）有了显著的改进。该引擎通过将多个代理的工作批处理到单个CUDA内核中来实现这一点，从而优化了内存带宽使用并降低了加载模型权重的成本。 AI

影响使得在消费级硬件上更高效地本地部署多个AI代理成为可能。

排序理由用于运行现有模型的新软件/引擎发布。

在 r/LocalLLaMA 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

OpenJet引擎使4个编码代理在同一块RTX 3090上以290 tok/s运行

本文如何被排名

Signal score: 6 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 用于运行现有模型的新软件/引擎发布。
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: infra, product; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: Breaking (< 6h); Fresh story with cross-source coverage still developing. Ranking may shift as more sources report.

完整方法见我们的编辑标准。

报道来源 [1]

r/LocalLLaMA TIER_1 English(EN) · /u/Adorable_Weakness_39 · 2026-10-27 13:15

4 coding agents on one RTX 3090: ~290 tok/s total from a custom CUDA engine for Qwen3.8-27B (3-4x llama-server --parallel)

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1xgs8hp/4_coding_agents_on_one_rtx_3090_290_toks_total/"> <img alt="4 coding agents on one RTX 3090: ~290 tok/s total from a custom CUDA engine for Qwen3.8-27B (3-4x llama-server --parallel)" src="https://exte…

本文如何被排名

报道来源 [1]

4 coding agents on one RTX 3090: ~290 tok/s total from a custom CUDA engine for Qwen3.8-27B (3-4x llama-server --parallel)

相关实体

相关话题