New benchmark reveals demographic bias in LLM quote attribution

A new benchmark dataset called AttriBench has been developed to investigate attribution bias in large language models (LLMs). This dataset, which is balanced for author fame and demographics, reveals significant disparities in quote attribution accuracy across racial, gender, and intersectional groups. The research also identified "suppression," a failure mode where LLMs omit attribution entirely, and found this bias is unevenly distributed, highlighting representational fairness issues in current models. AI

IMPACT Highlights critical representational fairness issues in LLMs, impacting their use in information retrieval and search applications.

RANK_REASON Research paper published on arXiv detailing a new benchmark dataset and findings on LLM bias. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• arXiv
• AttriBench
• Eliza Berman
• Hugging Face
• large-language models

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →