I'm still surprised on how good the kv quantization has become
A Reddit user expressed surprise at the effectiveness of KV quantization, noting its ability to accurately retrieve information from a 100,000-token context even at a Q4_0 quantization level. The user shared screenshots demonstrating this capability, with one example referencing obscure knowledge from a 2026 book, suggesting the model's performance extends beyond common training data. AI
