Prompting language influences diagnostic reasoning and accuracy of large language models
A new study published on arXiv reveals that the language used to prompt large language models significantly impacts their diagnostic reasoning and accuracy in clinical settings. Researchers found that four out of five evaluated models performed better when prompted in English compared to French, with English yielding higher scores in differential diagnosis, logical structure, and internal validity. Only one model, o3, showed no significant language-based performance difference, highlighting the need to consider linguistic and cultural factors for equitable global deployment of LLMs in healthcare. AI
IMPACT Highlights potential disparities in LLM clinical decision support based on language, impacting equitable access to AI healthcare tools.