MoE models tested for local coding on older GPU

This article evaluates Mixture-of-Experts (MoE) models for local coding tasks on a GTX 1080 Ti GPU. The author tested several MoE models, including Qwen3.8-Flash-Next and Qwen3.8-35B-A3B-Q4_K_M, assessing their perplexity on natural language and Go code datasets, as well as their generation speed using llama.cpp and llama-bench. The Qwen3.8-35B-A3B-Q4_K_M model unexpectedly performed best, completing a coding task in 15 minutes at 30 tokens/sec, significantly outperforming the larger Flash-Next model. AI

IMPACT Provides insights into optimizing MoE model performance on consumer-grade hardware for coding tasks.

RANK_REASON The article presents research findings on model performance and speed for a specific hardware configuration. [lever_c_demoted from research: ic=1 ai=1.0]

Read on Mastodon — mastodon.social →

• DeepSeek Harness
• GTX 1080 Ti
• llama-bench
• llama.cpp
• Qwen3.8-35B-A3B-Distill.APEX-I-MiniPlus-V2.1.gguf
• Qwen3.8-35B-A3B-Q4_K_M
• Qwen3.8-Flash-Next
• WikiText-2

• model release
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →