Ling-3.0 (BailingMoE3) model integrated into llama.cpp for local use

The Ling-3.0 (BailingMoE3) model has been officially integrated into the llama.cpp mainline, enabling its use in local setups. Quantized GGUF versions of the Ling-3.0 tiny (8B) and flash (127B) models are now available. Initial benchmarks on an Intel Arc B580 GPU show promising performance, with the 8B model achieving over 120 tokens/second and supporting contexts up to 128K within 12GB of VRAM. AI

IMPACT Enables broader local deployment and testing of the Ling-3.0 model.

RANK_REASON Integration of a specific model into an existing open-source inference engine.

Read on r/LocalLLaMA →

• Arc B580
• BailingMoE3
• Ling-3.0
• llama.cpp

• model release
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →