Mmq
PulseAugur coverage of Mmq — every cluster mentioning Mmq across labs, papers, and developer communities, ranked by signal.
1 day(s) with sentiment data
-
llama.cpp b11846 updates ggm-cuda backend for performance
The llama.cpp project has released version b11846, which includes updates to its ggm-cuda backend. This release enforces that the DP4A MMQ tile height is a multiple of the warp size, aiming to improve performance and co…
-
LLM inference on old hardware reveals evolving truths
The author details their experience running large language model inference on older hardware, drawing parallels to the evolving nature of scientific understanding. Initially, they held several assumptions about optimal …
-
Developer finds critical LLM config flag was dead code
A developer discovered that a widely shared environmental variable, `GGML_CUDA_FORCE_MMQ=1`, intended to optimize performance on Pascal GPUs like the Tesla P40, was actually dead code. This variable, frequently cited in…
-
llama.cpp CUDA pull request optimizes MMQ stream-k overhead for MoE models
A pull request to the llama.cpp project aims to reduce overhead in CUDA's MMQ stream-k operations. This optimization targets Mixture of Experts (MoE) models, potentially leading to faster prompt processing speeds. The c…