DeepSeek V4 benchmarks show 85 tok/s at 524k context; Ollama guide for Ryzen APUs released

New benchmarks reveal DeepSeek V4 Flash achieving 85 tokens per second with a 524k context window, utilizing MTP self-speculation and FP8 quantization on dual RTX PRO 6000 Max-Q GPUs. Additionally, a guide has been published for setting up Ollama with DeepSeek models on Ryzen APUs, making local LLM inference more accessible for users without dedicated graphics cards. A modified llama.cpp repository now supports Q4_K_M quantization for DeepSeek V4 Pro, further enabling local deployment. AI

影响 Demonstrates significant advancements in local LLM inference performance and accessibility for users with consumer hardware.

排序理由 Benchmark results for an open-weight model and a guide for local setup. [lever_c_demoted from research: ic=1 ai=1.0]

在 dev.to — LLM tag 阅读 →

• DeepSeek V4
• DeepSeek V4 Flash
• DeepSeek V4 Pro
• FP8 quantization
• llama.cpp
• MTP self-speculation
• Ollama
• Q4_K_M
• RTX PRO 6000 Max-Q GPUs
• Ryzen APU

• 模型发布
• 产品
• 基础设施

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →