LLM-Compressor
PulseAugur coverage of LLM-Compressor — every cluster mentioning LLM-Compressor across labs, papers, and developer communities, ranked by signal.
2 day(s) with sentiment data
-
Tiny Kimi-K3 Model Released for Testing on Hugging Face
A smaller, 0.4 billion parameter version of the moonshotai/Kimi-K3 model has been released on Hugging Face for testing and development purposes. This tiny model, named inference-optimization/Kimi-K3-0.40B, retains key a…
-
MedGemma-1.5-4B quantized to INT4 using llm-compressor
A technical guide details the process of quantizing Google's MedGemma-1.5-4B medical vision-language model to INT4 (W4A16) using the llm-compressor library. The author encountered and resolved several issues, including …
-
Stateful Transformers boost streaming inference; Intel releases AutoRound quantization toolkit
A new paper introduces a stateful transformer inference engine that significantly speeds up processing for streaming data by maintaining a persistent KV cache. This approach allows for query latency that is independent …