Researchers have investigated the information encoded within the hidden states of language models during chain-of-thought (CoT) reasoning. By using activation patching on the GSM8K dataset, they found that individual CoT tokens contain task-relevant information that can significantly improve answer accuracy when transferred to a direct-answer generation process. This task-solving information is more concentrated in correct CoT runs and is unevenly distributed across tokens, appearing earlier in the reasoning trace and in mid-to-late model layers. The study also revealed that language tokens are more crucial for steering correct reasoning, while mathematical tokens primarily encode answer-proximal content. AI
影响 Provides new insights into how language models represent and fail during reasoning, potentially guiding future model development.
排序理由 Academic paper analyzing the internal workings of language model reasoning.
AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →
