A new inference engine called Slipstream has been released, offering significant speed improvements for running the Qwen3.8-Flash-Next model on Apple Silicon Macs. Slipstream achieves 41-52 tokens/s on a 64GB Mac, a 1.76x increase compared to llama.cpp, and maintains this speed even with context lengths up to 130,000 tokens. Separately, users are reporting impressive performance with Qwen3.8-Flash-Next on multiple R9700 GPUs, with single streams reaching over 150 tokens/s. AI
IMPACT Accelerates local inference for large language models on consumer hardware, potentially enabling more complex applications on personal devices.
RANK_REASON Release of a new inference engine for a specific model, offering performance improvements.
AI-generated summary · Google Gemini · from 2 sources. How we write summaries →