New MELLA dataset enhances MLLMs for low-resource languages

Researchers have developed MELLA, a new multimodal dataset designed to improve the cultural grounding of Multimodal Large Language Models (MLLMs) in low-resource languages. Unlike translation-centric approaches, MELLA uses a dual-source strategy combining native image-alt-text pairs for cultural context and generated descriptions for linguistic richness. Experiments show that MELLA helps MLLMs recognize and articulate culturally specific entities, reducing hallucinations and enhancing understanding in languages with limited resources. AI

IMPACT This dataset could lead to more culturally aware and accurate AI interactions in a wider range of languages.

RANK_REASON The cluster describes a new academic paper introducing a dataset and methodology for improving AI models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• arXiv
• Hugging Face
• MELLA
• MLLMs
• Yufei Gao

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →