Gemma models on Apple Silicon suffer silent cache bug, slowing local AI

A cache bug affecting Gemma models on Apple Silicon has been identified, causing significant slowdowns in local AI agent performance. The issue stems from Gemma's sliding-window attention mechanism, which, when exceeding a certain context length, causes KV-cache reuse to fail silently. A fix involves changing the cache type to a plain KVCache, which restores performance without impacting model output quality, resulting in up to a 20x speed improvement for local coding agents. AI

IMPACT Fixes a silent performance bug in Gemma models on Apple Silicon, potentially speeding up local AI agents significantly.

RANK_REASON The item describes a bug fix and performance improvement for a specific local AI setup using Gemma models on Apple Silicon, rather than a new model release or major industry event.

Read on dev.to — LLM tag →

• Apple Silicon
• Claude Code
• claude-code-local
• Gemma
• Gemma 4
• Gemma 4.31B
• Mlx
• mlx-lm
• Qwen 3 Coder

• infra
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →