CuMA: Aligning LLMs with Sparse Cultural Values via Demographic-Aware Mixture of Adapters
Researchers have developed CuMA, a novel framework designed to align Large Language Models (LLMs) with diverse cultural values, addressing the issue of 'Mean Collapse' where models converge to a generic average. CuMA utilizes a 'Cultural Mixture of Adapters' approach, employing demographic-aware routing to separate conflicting gradients into specialized expert subspaces. Evaluations on benchmarks like WorldValuesBench and PRISM show CuMA significantly outperforms existing methods in preserving cultural diversity and mitigating mean collapse. AI
IMPACT This research offers a new method for LLM alignment that respects cultural diversity, potentially leading to more globally applicable AI systems.