Healthcare LLMs show significant cross-lingual factual disparities, paper finds

A new arXiv paper highlights significant disparities in the factual accuracy of Large Language Models (LLMs) when answering healthcare-related questions across different languages. Researchers developed a multilingual dataset from Wikipedia to analyze cross-lingual coverage and LLM response alignment. The study found that LLMs tend to align more closely with English Wikipedia content, even when prompted in other languages. However, providing contextual information from non-English Wikipedia sources during inference can improve factual alignment with culturally relevant knowledge, suggesting a path toward more equitable multilingual AI systems in healthcare. AI

IMPACT Highlights the need for improved multilingual capabilities in healthcare AI to ensure equitable access to information.

RANK_REASON The cluster contains a research paper published on arXiv detailing findings about LLM performance. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• arXiv
• English
• German
• Ipek Baris Schlicht
• Italian
• Large Language Models
• MultiWikiHealthCare
• retrieval-augmented generation
• Standard Chinese
• Turkish
• Wikipedia

• paper
• safety
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →