New method identifies demographic bias in LLM neurons

Researchers have developed a method called Fairness Pruning to identify and potentially mitigate demographic biases within large language models. This technique pinpoints specific neurons in GLU-MLP layers that react differently based on demographic attributes, using contrastive prompts and activation capture. Experiments on models like Llama-3.2 and Salamandra-2B showed that zeroing these identified neurons can alter the model's response to stereotypes, though it can lead to bidirectional bias destabilization rather than outright mitigation. The method is highly surgical, affecting a tiny fraction of model parameters while retaining significant reasoning and knowledge capabilities, suggesting that bias processing and core model functions utilize separable circuits. AI

IMPACT Introduces a precise method for dissecting and potentially controlling demographic bias in LLMs, paving the way for more targeted bias mitigation strategies.

RANK_REASON Academic paper detailing a new method for bias localization in LLMs. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• arXiv
• BiasScore
• Fairness Pruning
• GLU-MLP
• Hugging Face
• Llama-3.2
• Llama 3.2 1B
• Salamandra-2B

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →