New method improves multilingual text embeddings from frozen LLMs

Researchers have introduced Paired-Residual Subspace Removal (PRSR), a novel method to improve multilingual text embeddings from frozen large language models. Unlike previous approaches focused on English, PRSR addresses the misalignment of semantically equivalent sentences across languages by disentangling language-specific variations from semantic content. The technique involves constructing paired residuals from parallel sentences to identify language-specific deviations and then suppressing these components in the model's representations. Extensive evaluations across 18 languages on benchmarks like MTEB and MMTEB, as well as cross-lingual bitext mining, show that PRSR consistently enhances the performance of existing training-free embedding methods. AI

IMPACT Enhances multilingual capabilities of frozen LLMs, potentially improving cross-lingual search and understanding.

RANK_REASON The cluster contains an academic paper detailing a new method for improving multilingual text embeddings. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• arXiv
• English
• Hugging Face
• MMTEB: Massive Multilingual Text Embedding Benchmark
• MTEB
• Paired-Residual Subspace Removal

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →