Qwen2.5-Coder 32B and MoE models offer long context for local coding on high-end GPUs

A comparison of local coding LLMs for users with high-end GPUs like the RTX 4090 or 3090 highlights the trade-offs between model size, context window, and performance. Qwen2.5-Coder 32B is recommended for chat-length prompts and offers an 8K context window when quantized with AWQ and run via vLLM. For longer contexts, such as entire files or repositories, a Mixture-of-Experts (MoE) model like Qwen3-Coder 30B-A3B or North Mini Code 1.0 30B-A3B is suggested, as these models utilize fewer active parameters per token, allowing for larger context windows. AI

IMPACT Provides guidance on selecting local coding LLMs based on GPU VRAM and desired context length.

RANK_REASON Comparison of LLM models for local use on specific hardware.

Read on dev.to — LLM tag →

• 7900 XTX
• Activation Aware Quantization
• GGUF
• North Mini Code 1.0 30B-A3B
• Ollama
• RTX 3090
• RTX 4090
• vLLM

• model release
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →