A user benchmarked the Qwen3.8-Flash-Next 125B model using the MoEShift backend on two Apple Silicon Macs with 24GB of unified memory. The M5 Pro demonstrated superior performance, achieving 14.69 tokens/sec for short prompt decoding and 374.15 tokens/sec for long prompt prefill, while both machines maintained a peak memory usage of 8.5GB. This performance makes the model suitable for chat and agent applications, even with limited memory, and highlights the potential of MoEShift for running large models on memory-constrained devices. AI
IMPACT Demonstrates efficient large model performance on consumer hardware, potentially enabling more powerful local AI applications.
RANK_REASON User benchmark of a specific model version on particular hardware. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
