User builds AI rig from mining parts, achieves high inference speeds

A user on Reddit's r/LocalLLaMA subreddit shared their experience building a custom AI inference rig using repurposed mining hardware. The user detailed how they configured a system with three NVIDIA 3060 12GB GPUs, achieving impressive performance with the "flash next" software, specifically noting a throughput of 38-40 tokens per second on IQ3 through strata. This significantly outperforms llama.cpp's reported 13.2 tokens per second on the same hardware, prompting the user to inquire about others' success with similar dated mining equipment for AI tasks. AI

IMPACT Demonstrates potential for repurposing older hardware for efficient AI inference, potentially lowering costs for individuals.

RANK_REASON User-generated content about hardware configuration for AI inference.

Read on r/LocalLLaMA →

• FLASH
• llama.cpp
• NVIDIA 3060 12GB

• infra
• product

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →