Who Brought Easter Eggs to Eid? Auditing Cultural Translation of Math Word Problems Across Diverse Languages and Regions
A new study analyzed how large language models like Claude Opus 4, GPT-4.1, and Gemini 2.5 Pro translate math word problems across various languages and cultures. The research found that while models often agree on the type of transformation, they frequently substitute specific cultural elements like names and foods, leading to a significant divergence in the cultural context presented to students. Furthermore, all tested language-model combinations exhibited "entropy collapse," meaning the adaptation process compressed rather than expanded cultural diversity, and models often misattributed regional contexts or introduced cross-cultural contamination, such as equating egg hunts with Eid activities. AI
IMPACT Reveals significant limitations in LLMs' ability to perform nuanced cultural translation, impacting educational applications.