JetBrains enhances Mellum2.1 model with speculative multi-token prediction head

JetBrains has released Mellum2.1-12B-A2.5B-Thinking-MTP, a new version of their language model that incorporates a multi-token prediction (MTP) head. This additional transformer block, trained alongside the main model, speculatively drafts subsequent tokens, enabling inference engines to achieve 1.4–1.8x faster decoding speeds with minimal impact on output quality. The MTP head is compatible with various inference frameworks, including vLLM, Hugging Face's transformers library, and llama.cpp, with specific integration instructions provided for each. AI

IMPACT Accelerates inference speed for LLMs by enabling speculative token drafting, improving efficiency for applications using the Mellum2.1 model.

RANK_REASON The release is an update to an existing model with a new feature (MTP head), rather than a completely new frontier model release.

Read on r/LocalLLaMA →

• Hugging Face
• JetBrains
• llama.cpp
• Mellum2.1-12B-A2.5B-Thinking-MTP
• Mellum2.1 Thinking
• transformers
• vLLM

• model release
• product
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →