Multilingual Knowledge Transfer under Data Constraints via Lexical Interventions
Researchers have developed a novel data-level intervention method called LINK to enhance cross-lingual knowledge transfer in multilingual language models, particularly for languages with limited training data. This technique involves substituting words in the high-resource language (e.g., English) training corpus with their translations, using only a bilingual vocabulary. The method requires no additional model training or parallel data, significantly reducing the cost and complexity of improving performance on downstream tasks in low-resource languages. Evaluations across eight languages and five model sizes demonstrated notable improvements and up to a twofold training speedup to achieve equivalent performance. AI
IMPACT This method could significantly lower the barrier to creating high-performing multilingual models for languages with scarce data.