LLMs fail word counting due to specific MLP block, not representation limits

Researchers have identified a specific mechanism within large language models that causes them to fail at counting repeated words, despite correctly encoding the count in their internal representations. A multi-layer perceptron block, located around 85-93% of the network's depth, overwrites the accurate count with an incorrect one before it reaches the output. This phenomenon was observed consistently across different model sizes of Llama-3.2 and Qwen2.5, indicating a common architectural issue rather than a limitation in representation. AI

IMPACT Identifies a specific computational bottleneck in LLMs that could be targeted for improved reasoning capabilities.

RANK_REASON Academic paper detailing a specific finding about LLM behavior. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• arXiv
• Hugging Face
• Llama-3.2
• Qwen2.5
• Sohan Venkatesh

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →