ISTA GSQ-RCO has released quantized versions of the DeepSeek-V4.1-Flash-1017 model, offering GGUF formats at bit-widths ranging from 1.75 to 2.25 bits per weight. These quantized models maintain performance comparable to the original, with one version (IQ2_XXS) achieving 100% of the base model's benchmark scores. The release utilizes GSQ (Gumbel-Softmax Quantization) and RCO (Riemannian Constrained Optimization) techniques for efficient quantization, enabling the model to run on a single 12GB GPU. AI
IMPACT Provides more accessible versions of a large language model, potentially enabling wider use on consumer hardware.
RANK_REASON Release of quantized model weights and associated quantization techniques. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
