Users are reporting impressive performance gains with the Qwen 3.8 Flash Next model, particularly when utilizing the Basalt inference engine. One user on Reddit achieved significantly faster prefill speeds and better quality than previous models by switching to Basalt on a Linux system, ditching Windows due to its higher resource consumption. Another user demonstrated that the Qwen 3.8 Flash Next model can run at a sustained 22 tokens/second on more modest hardware, an RTX 3060 with 12GB VRAM and 16GB RAM, by developing a new fork of llama.cpp called Betelgeuse that optimizes expert routing and VRAM usage without sacrificing output quality. AI
IMPACT Demonstrates that significant performance improvements are achievable on consumer hardware with optimized inference engines and model configurations.
RANK_REASON User-developed optimizations and performance reports for an existing model.
- 16GB DDR4 RAM
- 64gb RAM
- Basalt
- Betelgeuse
- GeForce RTX 3060
- llama.cpp
- Microsoft Windows
- Qwen 3.6 35B
- Qwen 3.8 Flash Next
- Qwen 4
- Strata
AI-generated summary · Google Gemini · from 2 sources. How we write summaries →
