A new analysis of 117 local LLM rigs reveals that used RTX 3090 GPUs offer significantly better value for money than newer models like the RTX 5090. Despite the RTX 5090 being faster, the older RTX 3090 provides approximately three times more tokens per euro. For models fitting within 16GB of VRAM, cards like the RTX 5070 Ti and 5060 Ti offer competitive performance per euro. The study also highlights that speculative decoding techniques can provide a substantial speedup without requiring new hardware. AI
IMPACT Suggests that older, used GPUs may offer superior cost-efficiency for running local LLMs compared to newer, more expensive hardware.
RANK_REASON Analysis of hardware performance and value for local LLMs based on a dataset of user builds. [lever_c_demoted from research: ic=1 ai=0.7]
- AMD
- Apple Inc.
- Gemma
- GitHub
- GPT OSS 20B
- Intel
- llama.cpp
- Mlx
- Nemotron
- Nvidia
- Nvidia GeForce RTX 5070 Ti
- Ollama
- Qwen
- RTX 3090
- RTX 5060 Ti
- RTX 5090
- vLLM
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →