A user on r/LocalLLaMA has detailed their experience running the Nemotron-3-Puzzle-75B-A9B model with NVFP4 quantization across three NVIDIA 3090 GPUs. The setup achieved 132 tokens/second with a 256K context window and FP8 KV cache, consuming approximately 500W. This configuration effectively utilizes the available VRAM, offering dense-class quality at speeds comparable to smaller models, a niche that the user notes is otherwise underserved in the current market. AI
IMPACT Demonstrates efficient use of consumer hardware for large models, potentially influencing future model design and hardware optimization.
RANK_REASON User-reported performance details of a specific model on consumer hardware, not a formal release or benchmark from the model's creators.
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
