Qwen3 LLM runs up to 4.52x faster on Apple Silicon with ExecuTorch MLX delegate

A recent technical exploration demonstrates significant speed improvements when running the Qwen3-0.6B language model on Apple Silicon using ExecuTorch's experimental MLX delegate. The MLX delegate, which leverages Apple's MLX framework, achieved up to 4.52x faster decode throughput compared to PyTorch's MPS backend when using 4-bit quantization (INT4). While this quantization method drastically reduced file size and boosted speed, it was observed to alter the model's output in a small number of test cases. AI

IMPACT Demonstrates a path to significantly faster LLM inference on consumer hardware through optimized runtimes and quantization.

RANK_REASON Technical exploration of optimizing LLM performance on specific hardware using a new delegate. [lever_c_demoted from research: ic=1 ai=1.0]

Read on dev.to — LLM tag →

• Apple M1 Max
• Apple Silicon
• Executorch
• Mlx
• PyTorch
• Qwen3
• Qwen3-0.6B
• Qwen team

• infra
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →