Mapping Geopolitical Bias in 11 Large Language Models: A Bilingual, Dual-Framing Analysis of U.S.-China Tensions
A new research paper published on arXiv analyzes geopolitical bias in 11 large language models, focusing on U.S.-China tensions. The study developed a novel quantitative instrument by adapting survey psychometric techniques to measure model stance. This method involves posing propositions and their reversed counterparts to cancel out simple acquiescence, thereby isolating genuine conviction. The findings indicate that developer origin, query language, and issue domain are significant factors influencing bias, with all models, including those developed in the United States, showing a pro-China leaning when queried in Mandarin. AI
IMPACT This research provides a novel, reproducible method for quantifying geopolitical bias in LLMs, potentially influencing future model development and evaluation standards.