The llama.cpp project has released version b11763, introducing several new OpenCL variants for GEMM operations, particularly for small batch sizes. These updates include fixes for out-of-bounds reads and the removal of a condition that previously forced CPU weight placement. The release also adds specific optimizations for various quantization formats like Q4_0 and Q4_K, and includes support for multiple operating systems and hardware architectures, such as Apple Silicon, Linux, Android, and Windows. AI
IMPACT Improves performance and compatibility for running large language models on various hardware and operating systems.
RANK_REASON This is a software release for a tool that facilitates running LLMs, not a core AI model release or research paper.
Read on llama.cpp — Releases →
- ACL Graph
- Adreno
- android
- Apple Silicon
- dp4a
- GEMM
- gemv_noshuffle_q1_0_f32
- iOS
- KleidiAI
- Linux
- llama.cpp
- macOS
- Microsoft Windows
- mul_mm_f16_f32_l4_lm
- mul_mv_mxfp4_f32_flat
- OpenCL
- OpenEuler
- Q4_0
- Q4_K
AI-generated summary · Google Gemini · from 2 sources. How we write summaries →