Voice assistant uses CPU for ASR/TTS, freeing GPU for LLM

A user has successfully implemented a voice assistant that runs speech recognition and text-to-speech models on a CPU, freeing up the GPU for the primary LLM. They tested the Qwen3-ASR and Kokoro-TTS ONNX models on a 2022 Macbook M2 and an AMD Ryzen 9 7900, finding the M2 to be mostly usable and the Ryzen 9 to be very fast. The user has shared the code for others to test and explore possibilities with this setup. AI

IMPACT Enables more efficient use of hardware for local voice assistants by offloading ASR/TTS to CPU.

RANK_REASON User-level implementation of existing models for a specific use case.

Read on r/LocalLLaMA →

• AMD Ryzen 9 7900
• central processing unit
• Kokoro TTS
• Macbook M2
• ONNX
• Qwen3 ASR

• infra
• product

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →