Study finds function vectors in LLMs are largely language-agnostic for translation

Researchers have investigated whether function vectors (FVs), which represent tasks extracted from model activations during in-context learning, are language-agnostic. Using machine translation as a case study across three multilingual LLMs, they found that translation FVs from an English to another language direction improved translation accuracy for multiple unseen languages. The study also indicated that these FVs encode a largely language-agnostic translation signal rather than a language-pair-specific mapping, as top-performing tokens and ranked heads were shared across various languages. AI

IMPACT Suggests potential for more efficient multilingual model training and transfer learning.

RANK_REASON Academic paper detailing novel research findings on LLM function vectors. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• decoder-only multilingual LLMs
• English
• Function Vectors
• Machine Translation

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →