PulseAugur
EN
LIVE 08:47:46

New HorizonBench benchmark tests AI's ability to track evolving user preferences

Researchers have introduced HorizonBench, a new benchmark designed to evaluate how well AI personal assistants can adapt to users' evolving preferences over long periods. Unlike previous benchmarks, HorizonBench simulates real-world conversations where user preferences change due to life events, without explicit user statements about these changes. The benchmark includes a dataset of over 4,000 questions and a human longitudinal preference dataset to track preference shifts. Evaluations showed that current frontier models struggle to adapt, often selecting outdated preferences, and that explicit statements about preference changes significantly improve performance, highlighting a gap between remembering past information and understanding current user needs. AI

IMPACT Highlights a critical gap in current AI assistants' ability to adapt to long-term user preference changes, potentially driving research into more dynamic personalization.

RANK_REASON The cluster contains a research paper detailing a new benchmark and dataset for evaluating AI capabilities. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →

New HorizonBench benchmark tests AI's ability to track evolving user preferences

How we ranked this

Signal score
20 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Tool
The cluster contains a research paper detailing a new benchmark and dataset for evaluating AI capabilities. [lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration
Single-source cluster
Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics
paper, other
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
Breaking (< 6h)
Fresh story with cross-source coverage still developing. Ranking may shift as more sources report.

Full methodology in our editorial standards.

COVERAGE [1]

  1. arXiv cs.AI TIER_1 English(EN) · Shuyue Stella Li, Bhargavi Paranjape, Kerem Oktar, Zhongyao Ma, Gelin Zhou, Lin Guan, Na Zhang, Sem Park, Lin Chen, Diyi Yang, Yulia Tsvetkov, Asli Celikyilmaz ·

    HorizonBench: Long-Horizon Personalization with Evolving Preferences

    arXiv:2604.17283v2 Announce Type: replace-cross Abstract: Personal assistants now talk with the same user for months, and users' preferences change as their lives change; an assistant that keeps serving a preference the user has outgrown fails that user. Real conversations do not…