Mi50 32GB / GFX906 - vLLM Qwen 3.5 Configuration for Qwen 3.5:9B AWQ-4bit
A user is seeking assistance with configuring the Qwen 3.5 9B model for optimal local inference on a MI50 32GB GPU. They are experiencing slow speeds, below 1 token per second, while using a specific vLLM fork. The user is looking for guidance to improve performance and potentially set up a vision/text-to-text model or a Gemma 4 variant. AI
IMPACT This query highlights challenges in optimizing local LLM inference, particularly with specific hardware and model configurations.