Consumer GPUs achieve high LLM speeds with 122B model running at 37 t/s

A user on Reddit's r/LocalLLaMA subreddit shared impressive benchmarks for running large language models on consumer hardware. They achieved 206 tokens per second with a 35 billion parameter model (35b a3b) using an Nvidia RTX 4090 and an RTX 5060 Ti, with some layers offloaded to 64GB of system RAM. Even more remarkably, a 122 billion parameter model (barium-122) was run at 37 tokens per second under similar conditions, exceeding the user's optimistic expectations. AI

IMPACT Demonstrates significant advancements in running large language models on consumer-grade hardware, potentially lowering barriers to entry for AI experimentation.

RANK_REASON User-generated benchmark of consumer hardware running LLMs.

Read on r/LocalLLaMA →

• 35b a3b
• 4090
• 64Gb RAM
• barium-122
• Claude
• llama.cpp
• Nvidia RTX 5060 Ti
• Qwen3.5-122B-A10B
• Qwen3.6-27B

• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →