LLM clinical accuracy varies significantly by prompting language, study finds

A new study published on arXiv reveals that the language used to prompt large language models significantly impacts their diagnostic reasoning and accuracy in clinical settings. Researchers found that four out of five evaluated models performed better when prompted in English compared to French, with English yielding higher scores in differential diagnosis, logical structure, and internal validity. Only one model, o3, showed no significant language-based performance difference, highlighting the need to consider linguistic and cultural factors for equitable global deployment of LLMs in healthcare. AI

Summary written by gemini-2.5-flash-lite from 1 source. How we write summaries →

IMPACT Highlights potential disparities in LLM clinical decision support based on language, impacting equitable access to AI healthcare tools.

RANK_REASON Academic paper detailing model performance evaluation. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• BioMistral-7B
• DeepSeek-R1
• GPT-4-Turbo
• Llama-3.1-405B-Instruct
• LLMs
• o3

• paper
• safety