The author conducted a week-long performance benchmark of the Muse Glimmer 30B model running on AMD's Strix Halo platform, specifically the Ryzen AI Max+ 395 chip. The tests, performed on Linux with ROCm, explored various configurations including context depth, serving shapes, KV cache types, and quantization levels. Key findings indicate that optimal draft length for the model varies significantly depending on the task, with agent turns benefiting from longer draft settings (n-max 15-16) for higher token-per-second throughput compared to open prose generation. AI
IMPACT Provides detailed performance insights for running LLMs on AMD's Ryzen AI hardware, informing optimization strategies for local AI deployments.
RANK_REASON Detailed technical benchmark of an LLM on specific hardware. [lever_c_demoted from research: ic=1 ai=1.0]
- AMD
- gfx1151
- GMKtec EVO-X2
- Linux
- llama.cpp
- Radeon 8060S
- ROCm
- Ryzen AI MAX+ 395
- Strix Halo
- unsloth/Muse-Glimmer-30B-GGUF
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →