Value Entanglement: Conflation Between Different Kinds of Good In (Some) Large Language Models
A new research paper explores how large language models (LLMs) conflate different types of "good," specifically moral, grammatical, and economic values. Researchers found that LLMs tend to overemphasize moral considerations in grammatical and economic contexts, deviating from human norms. This "value entanglement" was observed by analyzing model behavior and embeddings, and the study demonstrated that selectively removing moral activation vectors could repair this conflation. AI
IMPACT Reveals potential biases in LLMs that could affect their application in diverse domains, highlighting the need for more nuanced value alignment.