New benchmark tests LLM safety in African languages

Researchers have developed TukaBench, a new benchmark designed to evaluate the safety of large language models (LLMs) in seven African languages. This benchmark goes beyond simple translation by incorporating culturally adapted prompts, human-curated prompts validated with GPT-5.2, and code-switched prompts. Initial findings indicate that LLMs are less likely to refuse prompts in African languages compared to English, with culturally specific prompts showing the lowest refusal rates. The study also highlighted challenges in LLM comprehension and reliability as judges in these lower-resource languages. AI

IMPACT This benchmark is crucial for improving LLM safety and reliability in underrepresented languages, pushing for more equitable AI development.

RANK_REASON The cluster contains an academic paper introducing a new benchmark for LLM safety evaluation. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• GPT-5.2
• LLMs
• TukaBench
• Victor Akinode Engr.

• safety
• paper

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →