Researchers have introduced HorizonBench, a new benchmark designed to evaluate how well AI personal assistants can adapt to users' evolving preferences over long periods. Unlike previous benchmarks, HorizonBench simulates real-world conversations where user preferences change due to life events, without explicit user statements about these changes. The benchmark includes a dataset of over 4,000 questions and a human longitudinal preference dataset to track preference shifts. Evaluations showed that current frontier models struggle to adapt, often selecting outdated preferences, and that explicit statements about preference changes significantly improve performance, highlighting a gap between remembering past information and understanding current user needs. AI
IMPACT Highlights a critical gap in current AI assistants' ability to adapt to long-term user preference changes, potentially driving research into more dynamic personalization.
RANK_REASON The cluster contains a research paper detailing a new benchmark and dataset for evaluating AI capabilities. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →