Disentangling Linguistic Relatedness from Task Alignment in Cross-Lingual Transfer
A new research paper explores cross-lingual transfer in large language models, specifically examining Arabic fine-tuning and its impact on Semitic languages. The study found no evidence of Semitic-specific transfer, indicating that improvements in models are more related to task-format alignment than genuine cross-lingual knowledge transfer. This was observed across various model sizes and architectures, including Mixture-of-Experts models. AI
IMPACT Suggests current LLM fine-tuning methods may not effectively transfer knowledge across related languages, highlighting task alignment as a key factor.