Code-Switching Reveals Language Anchoring in Multilingual LLMs
Researchers have developed a new method called CANVAS to improve the performance of multilingual large language models (MLLMs) when processing code-switched inputs. By analyzing "Anchor Bias," a measure of how closely a model's internal states align with source or target languages, they identified a consistent grammar-frame effect that degrades question-answering performance. CANVAS, an inference-time intervention, steers target-language hidden states toward the source anchor, consistently recovering question-answering F1 scores across various MLLMs and code-switching conditions. AI
IMPACT This research offers a novel approach to enhance the robustness of multilingual LLMs in handling code-switched inputs, potentially improving their usability in diverse linguistic contexts.