SemiAnalysis reports that the GB300 NVL72, utilizing an innovative copper backplane and NVLink, offers a 13x improvement in performance per dollar compared to NVIDIA's Hopper architecture for agentic inference. This advanced design enables a scale-up domain of 72 GPUs, significantly larger than Hopper's 8, facilitating optimizations like vLLM's wide expert parallelism. AI
IMPACT This hardware advancement could significantly reduce the cost of running large AI models, particularly for agentic inference tasks.
RANK_REASON The item discusses a new hardware architecture and its performance benefits, which falls under research into hardware capabilities. [lever_c_demoted from research: ic=1 ai=0.7]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →