hipEngine: Fast Native Qwen 3.6 Inference for RDNA3 (Strix Halo, 7900 XTX)
A new open-source inference engine called hipEngine has been developed for AMD's RDNA3 GPUs, enabling faster native inference of the Qwen 3.6 large language model. The engine, written in Python with a HIP/C++ core, utilizes AMD's native libraries to achieve competitive performance against llama.cpp. Benchmarks show hipEngine outperforming llama.cpp in prompt processing speeds across various context lengths, particularly at 128K context, and demonstrating lower peak memory usage. AI
IMPACT Enables faster local LLM inference on AMD GPUs, potentially broadening hardware accessibility for AI model deployment.