Developers are discussing their experiences with the Mistral Medium 3.5 (128b dense) model, particularly its performance when quantized to 3-bit. One user reported that a 3-bit quantized version of the model, despite initial reservations about the quantization level, performed well on their project, identifying issues their usual model missed. While acknowledging it is slower than Mixture-of-Experts models, the user found its speed acceptable for their needs, achieving around 8 tokens/sec with specific KV cache and quantization settings. AI
IMPACT Provides user insights into the performance and usability of a specific large dense model, informing developer choices.
RANK_REASON User discussion and anecdotal experience with a specific model version, not an official release or benchmark.
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
