PulseAugur
EN
LIVE 21:40:14

Small LLMs use positional copying shortcut for arithmetic, bypassing CoT logic

By PulseAugur Editorial · [2 sources] ·

A new research paper reveals a significant shortcut in how small language models perform arithmetic tasks using chain-of-thought (CoT) prompting. Instead of relying on logical sequencing, these models tend to copy the number positioned just before the answer delimiter, regardless of the intermediate reasoning steps. This positional copying accounts for a large portion of their accuracy, even when the preceding steps are incorrect or shuffled, highlighting a potential failure mode in evaluating CoT faithfulness. AI

IMPACT Reveals a critical flaw in evaluating arithmetic reasoning in small LLMs, suggesting current faithfulness evaluations may be misleading.

RANK_REASON The cluster contains an academic paper detailing a novel finding about the behavior of language models.

Read on Hugging Face Daily Papers →

AI-generated summary · Google Gemini · from 2 sources. How we write summaries →

COVERAGE [2]

  1. arXiv cs.AI TIER_1 English(EN) · Ming Liu ·

    The Readout Shortcut: Positional Number Copying Dominates Arithmetic CoT Readout in Small Language Models

    arXiv:2605.22870v1 Announce Type: cross Abstract: Chain-of-thought (CoT) prompting is necessary for arithmetic in small language models, yet shuffling its steps preserves most performance. What does CoT contribute if not logical sequencing? In three 1-3B instruction-tuned LMs on …

  2. Hugging Face Daily Papers TIER_1 English(EN) ·

    The Readout Shortcut: Positional Number Copying Dominates Arithmetic CoT Readout in Small Language Models

    Chain-of-thought (CoT) prompting is necessary for arithmetic in small language models, yet shuffling its steps preserves most performance. What does CoT contribute if not logical sequencing? In three 1-3B instruction-tuned LMs on GSM8K, we isolate the answer-readout stage via pre…

RELATED ENTITIES

RELATED TOPICS