GPU benchmarks reveal local LLM performance for Qwen, Gemma, and gpt-oss

This article provides measured generation speeds for several open-source large language models, including Qwen3.8, Qwen3.6, Gemma 4, and gpt-oss, across various GPUs. It details the performance in tokens per second for different model sizes and architectures (dense vs. Mixture-of-Experts), highlighting the impact of factors like memory bandwidth, quantization, and speculative decoding. The analysis aims to help users determine which hardware is suitable for running these models locally, offering recommendations based on specific model requirements and GPU capabilities. AI

IMPACT Provides practical guidance for users on hardware requirements for running various open-source LLMs locally.

RANK_REASON Article provides measured performance data for running LLMs locally on consumer hardware, serving as a practical guide for users.

Read on dev.to — LLM tag →

• 4090
• Activation Aware Quantization
• DGX Spark
• Gemma 4
• Giles Thomas
• gpt-oss
• Jeff Geerling
• llama.cpp
• Multi Token Prediction
• OpenJet
• Qwen3.6
• Qwen3.8
• RigBase
• RTX 3090
• Sleeping Robots
• vLLM

• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →