DeepSeek V4 Flash performance tested: IQ2_S faster than IQ3_XXS-AS

A user conducted performance tests on the DeepSeek V4 Flash model, comparing two quantization levels (IQ3_XXS-AS and IQ2_S) using the fairydreaming/llama.cpp fork. The results showed that the IQ2_S quantization was faster for both initial prompt evaluation and subsequent generation, despite using a smaller context window. The user also found that the fairydreaming fork offered no performance advantage over the mainline llama.cpp build for this model. AI

IMPACT Provides practical performance data for users running DeepSeek V4 Flash on consumer hardware, aiding model selection.

RANK_REASON User-conducted benchmark of an open-source model. [lever_c_demoted from research: ic=1 ai=1.0]

Read on r/LocalLLaMA →

• DeepSeek-V4 Flash
• fairydreaming/llama.cpp
• IQ2_S
• IQ3_XXS-AS
• mainline llama.cpp
• RTX 3090

• model release
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →