一位Reddit用户正在寻求关于构建本地推理服务器的建议，预算为15万美元。他们目前的生产服务器使用四块H100 GPU，并且正在寻找一个相当或更好的替代品，考虑到H100即将结束产品周期。用户优先考虑推理的成本效益，并且需要服务器能够处理像122b AWQ这样在256k上下文长度下TP为2的大模型，以及一个小型的嵌入模型。 AI
排序理由 用户在论坛上生成的内容，寻求建议，不是新闻报道。
AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →
用户寻求15万美元本地推理服务器建议
本文如何被排名
- Signal score
- 0 / 100
- Composite score across the factors below. Higher = stronger signal that this story matters right now.
- Newsworthiness bucket
- Meme
- 用户在论坛上生成的内容，寻求建议，不是新闻报道。
- Source corroboration
- Single-source cluster
- Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
- Topics
- infra
- Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
- AI-industry relevance
- High
- Clearly on-topic for AI-industry coverage.
- Story freshness
- 92 days old
- Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.
完整方法见我们的编辑标准。
报道来源 [1]
-
如果你有15万美元用于构建一个能服务300人的生产级本地推理服务器，你会买什么？
<!-- SC_OFF --><div class="md"><p>I know we usually focus on home lab stuff here for the most part, but I’m in a position where I’m trying to purchase a failover server for our production inference server for under $150K. Our main production server has 4 H100s, so I’m looking for…