235B Qwen3 model runs on 48GB MacBook via custom C++ engine

A developer has successfully run the 235-billion-parameter Qwen3-235B-A22B-Instruct-2507 model on a consumer MacBook with 48 GB of RAM. This was achieved by using a custom C++ engine and Metal kernels, streaming the model's experts from the Solid State Drive. The process was slow and imperfect, but demonstrated that large frontier models can operate on consumer hardware, challenging the assumption that they require massive GPU clusters. A key debugging challenge involved a mismatch in the chat template, which was resolved by loading the correct tokenizer. AI

IMPACT Proves that large frontier models can be run on consumer hardware, potentially democratizing access and use.

RANK_REASON Demonstrates running a large frontier model on consumer hardware, which is a research-level achievement. [lever_c_demoted from research: ic=1 ai=1.0]

Read on dev.to — LLM tag →

• CPP
• MacBook
• Metal
• Qwen3-235B-A22B-Instruct-2507
• Sharon Moeno

• model release
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →