The NUMALLAMA project has released an update that significantly boosts prompt processing speeds, particularly for Llama 3.1 8B Q8 models, which now achieve around 800 tokens/sec. This update also introduces ready-made models on Hugging Face, including Qwen3-30B-A3B, Qwen3.8-27B, and Qwen2.5, compatible with OpenAI clients. The performance optimizations are specifically tuned for ARM64 architecture, with notable speed improvements observed on dual-socket ARM servers without GPUs. AI
IMPACT Improves LLM performance on ARM hardware, making it more accessible for users without GPUs.
RANK_REASON This is an update to an open-source project that improves performance and accessibility of LLMs on specific hardware, rather than a new frontier model release.
- Arm Holdings
- Hugging Face
- KleidiAI
- Llama 3.1
- Llama 3.2
- NUMALLAMA
- OpenAI
- Qwen2.5
- Qwen3-30B-A3B
- Qwen3.8-27B
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →