DeepSeek-V4-Flash IQ2_XS runs on single RTX 3090, impresses user

A user on Reddit's r/LocalLLaMA subreddit shared their experience running the DeepSeek-V4-Flash model, specifically the IQ2_XS quantization, on a single RTX 3090 graphics card. Despite the heavy quantization, the user was impressed with the model's ability to produce a complete and functional output for the Döner Bench test, noting that while some fine details were lost, the overall scene and concept were preserved. The post includes details on the hardware used, the prompt, and the specific command-line arguments employed for running the model via llama.cpp. AI

IMPACT Demonstrates the feasibility of running advanced LLMs on consumer-grade hardware with aggressive quantization.

RANK_REASON User-generated report on running a specific model quantization on consumer hardware.

Read on r/LocalLLaMA →

• DeepSeek-V4 Flash
• Döner Bench
• IQ2_XS
• llama.cpp
• RTX 3090

• model release
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →