A user on Reddit's r/LocalLLaMA subreddit is experiencing a significant reduction in context window size when using the Qwen 27B model with MTP (Multi-Turn Prompting) enabled. Despite the model's reported 137k context window, enabling MTP via specific command-line arguments (`spec-type draft-mtp --spec-draft-n-max 2`) causes the effective context size to drop to 14k. The user is seeking confirmation if this drastic reduction is expected behavior. AI
IMPACT Investigating potential limitations or bugs in local LLM deployments.
RANK_REASON User query about a specific model's behavior with a particular feature enabled.
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
