OpenAI's GPT OSS 20B leads in speed and coding benchmarks on Mac

A comparison of three open-weight LLMs—OpenAI's GPT OSS 20B, Alibaba's Qwen3 14B, and Mistral AI's Mistral-Small 24B—was conducted on an Apple M2 machine with 24GB of RAM. GPT OSS 20B emerged as the fastest, outperforming Qwen3 14B by three to four times, and achieved perfect scores on GSM8K and HumanEval+ benchmarks. Qwen3 14B, however, led in raw knowledge with an 82% MMLU score, surpassing GPT OSS 20B's 72%. Mistral-Small 24B struggled with memory limitations on the 24GB hardware, timing out on larger benchmarks and proving to be an unreliable choice for this specific configuration. AI

IMPACT Provides practical insights into the performance of open-weight LLMs on consumer hardware, guiding developers on model selection for resource-constrained environments.

RANK_REASON Comparison of open-weight LLMs on specific hardware and benchmarks. [lever_c_demoted from research: ic=1 ai=1.0]

Read on dev.to — LLM tag →

• Alibaba Group
• Apple M2
• GPT OSS 20B
• GSM8K
• HumanEval+
• IFEval
• Massive Multitask Language Understanding
• Mistral AI
• Mistral-Small 24B
• Ollama
• OpenAI
• Qwen3 14B

• model release
• product

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →