Muse-Glimmer 30B achieves 287 t/s with DFlash speculative decoding

The Muse-Glimmer 30B model, when paired with DFlash speculative decoding, achieved impressive performance metrics in real-world coding tasks. Running on a single RTX 5090 GPU with llama.cpp, the model demonstrated generation speeds of up to 287 tokens per second, with an average of around 173 tokens/sec for IDE completion. Prompt processing also showed strong results, handling 14.3k tokens in approximately 5.35 seconds, and the draft acceptance rate averaged 82.3%, indicating efficient multi-token acceptance. AI

IMPACT Demonstrates significant speed improvements for local LLMs in coding tasks, potentially accelerating developer workflows.

RANK_REASON The item details performance metrics and technical configurations of a specific open-source LLM, fitting the research category. [lever_c_demoted from research: ic=1 ai=1.0]

Read on r/LocalLLaMA →

• llama.cpp
• Next.js
• Qwen 3.6 27B
• RTX 5090

• model release
• product

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →