DeepSeek releases distilled R1 models for local AI inference

DeepSeek has released six distilled versions of its R1 reasoning model, designed for local AI deployment on consumer hardware. These smaller models, derived from the massive 671B parameter original, range from 1.1GB to 43GB and are built upon Qwen2.5 and Llama 3 architectures. The smallest variants can run on GPUs with as little as 8GB of VRAM, offering impressive performance on math and coding benchmarks that rival larger, older models. AI

IMPACT Enables local inference of advanced reasoning models on consumer hardware, democratizing access to powerful AI capabilities.

RANK_REASON Release of smaller, distilled versions of an existing model for local deployment. [lever_c_demoted from research: ic=1 ai=1.0]

Read on dev.to — LLM tag →

• AIME 2024
• DeepSeek
• RTX 3060
• GPT-3.5
• Llama 3
• MATH-500
• Qwen2.5
• R1
• RTX 3090
• RTX 4090

• model release
• product

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →