Researchers have identified "Bridge Routing Heads" (BRHs) within large multilingual LLMs, revealing how these models handle multi-hop reasoning across different languages. The study found that BRHs are largely language-specific, with minimal overlap between circuits for languages like English, Spanish, French, German, and Chinese in models such as Llama 3.1 70B and Qwen 2.5-72B. Ablating these heads significantly degraded reasoning performance, while amplifying them in failing instances improved accuracy by up to 51.7% without retraining, demonstrating the potential for activation-level interventions to correct cross-lingual reasoning errors. AI
IMPACT Identifies specific internal mechanisms for multilingual reasoning in LLMs, potentially guiding future model development and interpretability efforts.
RANK_REASON Research paper detailing a new finding about LLM internal mechanisms. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
