TukaBench: A Culturally Grounded Jailbreak Benchmark for African Languages
Researchers have developed TukaBench, a new benchmark designed to evaluate the safety of large language models (LLMs) in seven African languages. This benchmark goes beyond simple translation by incorporating culturally adapted prompts, human-curated prompts validated with GPT-5.2, and code-switched prompts. Initial findings indicate that LLMs are less likely to refuse prompts in African languages compared to English, with culturally specific prompts showing the lowest refusal rates. The study also highlighted challenges in LLM comprehension and reliability as judges in these lower-resource languages. AI
IMPACT This benchmark is crucial for improving LLM safety and reliability in underrepresented languages, pushing for more equitable AI development.