DeepSeek V4 benchmarks show 85 tok/s at 524k context; Ollama guide for Ryzen APUs released

New benchmarks reveal DeepSeek V4 Flash achieving 85 tokens per second with a 524k context window, utilizing MTP self-speculation and FP8 quantization on dual RTX PRO 6000 Max-Q GPUs. Additionally, a guide has been published for setting up Ollama with DeepSeek models on Ryzen APUs, making local LLM inference more accessible for users without dedicated graphics cards. A modified llama.cpp repository now supports Q4_K_M quantization for DeepSeek V4 Pro, further enabling local deployment. AI

IMPACT Demonstrates significant advancements in local LLM inference performance and accessibility for users with consumer hardware.

RANK_REASON Benchmark results for an open-weight model and a guide for local setup. [lever_c_demoted from research: ic=1 ai=1.0]

Read on dev.to — LLM tag →

• DeepSeek V4
• DeepSeek V4 Flash
• DeepSeek V4 Pro
• FP8 quantization
• llama.cpp
• MTP self-speculation
• Ollama
• Q4_K_M
• RTX PRO 6000 Max-Q GPUs
• Ryzen APU

• model release
• product
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →