LLMs show bias toward majority groups when prompted with demographics

A new study has revealed that large language models (LLMs) do not act as neutral judges when prompted with demographic information. Instead, models without any demographic conditioning tend to align with the judgments of White, college-educated annotators. The research further indicates that conditioning LLMs on demographic profiles can inadvertently move their judgments away from minority groups, particularly when using intersectional profiles that combine gender, age, race, and education. Instruction tuning appears to be a contributing factor to this observed asymmetry, suggesting that sociodemographic prompting should be used with extreme caution. AI

IMPACT Reveals potential biases in LLMs when using demographic conditioning, suggesting caution is needed for applications aiming to represent diverse viewpoints.

RANK_REASON The cluster contains a research paper detailing findings on LLM behavior. [lever_c_demoted from research: ic=1 ai=1.0]

Read on Hugging Face Daily Papers →

• Age
• College-educated women's personality development in adulthood: perceptions and age differences
• Education
• gender
• Instruction Tuning Text-to-SQL with Large Language Models in the Power Grid Domain
• Large language models
• minority group
• race
• Sociodemographic prompting
• White

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →