SemiAnalysis推出了一种评估大型语言模型（LLM）推理效率的新颖指标：每秒每兆瓦的代币数（tok/s/MW）。该指标允许对能源消耗与产出进行比较，计算显示，一个巨无霸汉堡所含卡路里理论上可以在B300 GPU配置上产生约18,000个代币。相比之下，同样的卡路里能量可以支持人类说话产生约400,000个代币，这凸显了当前LLM技术与人类生物过程在能源效率上的显著差异。 AI
影响 引入了评估LLM能源效率的新指标，突显了当前AI能力与人类生物能源效率之间的差距。
排序理由 该集群讨论了一种新颖的LLM效率指标，并将其与人类语音进行比较，但没有宣布新模型、研究突破或重大的行业事件。
AI 生成摘要 · Google Gemini · 来自 4 个来源。 我们如何撰写摘要 →
Unfortunately, the average human is much dumber than most frontier language models. So while we can speak to throughput per Big Mac, goodput might be a different story 😅 (4/4) https://t.co/FV2YZUeCPT
Finally, a Big Mac contains roughly 580 calories which means: 🟠 B300: 30.8 × 580 = ~18,000 output tokens per Big Mac 🟠 Human: 690 × 580 = ~400,000 output tokens per Big Mac One Big Mac can produce a lot of output tokens! (3/4) https://t.co/0LY7kD9rDT
Using the fact that 1 MW = 1,000,000 J/s and 1 joule = 0.000239 food Calories, we can convert power efficiency into tokens per Calorie: 7,368 tok/s/MW = 0.007368 tok/J 0.007368 ÷ 0.000239 = 30.8 output tok/Calorie Given that an average human speaks at ~3.3 tok/s while the http…
A useful metric for evaluating LLM inference is tok/s/MW: how many tokens a system generates per second for each megawatt of all-in provisioned power. At one concurrent user, the B300 configuration below generates ~14 output tok/s per GPU on DeepSeek V4. The SemiAnalysis https:/…