Response Time Enhances Alignment with Heterogeneous Preferences
Researchers have developed a new method to improve the alignment of large language models with human preferences by incorporating response times into preference datasets. This approach addresses the limitation of standard methods that assume uniform preferences among labelers, which can distort the learned model policy. By modeling decisions using a Drift-Diffusion Model, the new technique can identify the population's average preference even with heterogeneous and anonymous feedback, outperforming existing baselines. AI
IMPACT Enhances LLM alignment by incorporating response times, potentially improving model safety and utility with diverse user groups.