Language models fail to transfer reasoning states via direct activation injection

Researchers have investigated whether one language model can directly transfer its internal reasoning states to another model during inference. While a linear translation layer successfully mapped hidden states between Pythia models with high similarity, injecting these translated activations did not improve the receiver model's performance. The study found that both low-strength additive injection and replacement-style injection were ineffective, indicating that offline representational alignment alone is insufficient for causal communication between models in this specific setting. AI

IMPACT Demonstrates limitations in direct inter-model communication, suggesting current methods for transferring learned reasoning are insufficient.

RANK_REASON The cluster contains a research paper detailing experimental results. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• arXiv
• Pythia

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →