llama.cpp adds ARM NVFP4 dot product optimization via UE4M3 LUT

A pull request for the llama.cpp project introduces an optimization for ARM processors by extending the UE4M3 lookup table (LUT) to the NVFP4 dot product implementation. This change aligns the ARM version with existing x86 optimizations, leveraging a shared LUT infrastructure. Benchmarking shows a significant performance increase, with one test case improving from 1.89 tokens/second to 9.97 tokens/second on a Qwen3.5-4B-NVFP4 model using 4 threads. AI

IMPACT Improves inference speed for specific models on ARM-based systems.

RANK_REASON This is a pull request for a specific optimization within an open-source project, not a new model release or significant industry event.

Read on r/LocalLLaMA →

• ggml-cpu
• ggml-org
• llama.cpp
• NVFP4
• Qwen3.5-4B-NVFP4
• ragz4125
• UE4M3

• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →