PAFO: Pareto Fairness Optimization for Personalized Reward Modeling
Researchers have introduced PAFO, a new framework designed to address personalized reward bias in large language models. This bias occurs when reward models, trained on diverse user preferences, disproportionately favor users with more common preferences. PAFO formulates fairness as a Pareto optimization problem, aiming to enhance the experience for under-served users without negatively impacting others. The framework trains specialized models for different user groups and then distills their knowledge into a single model, improving accuracy and fairness across the board. AI
IMPACT Addresses fairness issues in LLM personalization, potentially leading to more equitable user experiences.