A user on Reddit shared their experience running the Qwen 3.6 35B model on a consumer-grade setup, including an RTX 3080 GPU and 32GB of RAM. They achieved a throughput of 26 tokens/second for generation and 1400 tokens/second for processing at a 32k context length. While offloading the KV cache to the GPU boosted generation speed to 56 tokens/second, it limited the context window, which was not suitable for their agentic work involving deep research and document processing. AI
IMPACT Provides a performance data point for running large models locally, informing users about achievable speeds and context lengths on consumer-grade GPUs.
RANK_REASON User-shared benchmark and configuration details for a specific model on consumer hardware. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →