LLMs struggle with Bangla medical visual questions, new dataset shows

Researchers have developed BanglaMedVQA, a new dataset designed to evaluate Large Language Models (LLMs) and Large Vision Language Models (LVLMs) on medical visual question answering in the Bangla language. Their benchmarking reveals that even leading models like Gemini and GPT-4.1 mini struggle significantly with diagnostic questions in Bangla, highlighting the challenges of low-resource languages in specialized domains. While some open-source models show promise in general categories, they also fail on clinically complex queries, indicating a need for improved evaluation methods and model capabilities. AI

影响 Highlights significant limitations of current LLMs in handling specialized medical queries in low-resource languages, indicating a need for improved multilingual and domain-specific reasoning capabilities.

排序理由 The cluster contains an academic paper introducing a new dataset and benchmarking results for LLMs on a specific task. [lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.CV 阅读 →

• BanglaMedVQA
• Gemini
• Gemma-3
• GPT-4.1 mini
• LLMs
• LVLMs
• Mir Sazzat Hossain

• 论文
• 模型发布

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →