Local LLM Costs Revealed: Smaller Models Cheaper Than Cloud, Larger Ones More Expensive

A controlled benchmark on a single machine with an RTX 3090 GPU measured the actual cost of running local Large Language Models (LLMs) in euros per million tokens. The results indicated that smaller models like Gemma 3:1B were significantly cheaper than hosted APIs, costing approximately €0.118 per million tokens. However, larger models such as Gemma 3:27B proved more expensive to run locally due to high energy consumption and lower throughput, costing €0.706 per million tokens, before accounting for hardware depreciation. AI

IMPACT Highlights that the cost-effectiveness of running LLMs locally is highly dependent on model size and hardware efficiency, challenging the assumption that local deployment is always cheaper.

RANK_REASON The item details a specific, reproducible benchmark and analysis of LLM operational costs, akin to a research finding. [lever_c_demoted from research: ic=1 ai=1.0]

Read on dev.to — LLM tag →

• Gemma 3:1B
• Gemma 3:27B
• LLM
• ollama
• RTX 3090

• infra
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →