New benchmark tests LLMs on dynamic user profiling from streaming content

By PulseAugur Editorial · [3 sources] · 2026-05-25 12:10

A new benchmark, StreamProfileBench, has been introduced to evaluate how well Large Language Models (LLMs) can infer user profiles from continuously arriving content, a task that current static evaluations overlook. The benchmark includes a dataset of over 120,000 user-generated content posts from 7,000+ real users across five platforms. Experiments with 14 LLMs revealed that models struggle with continuous profile updating, exhibiting a bias towards retaining old interests and failing to recognize decaying ones. AI

IMPACT Highlights limitations in LLMs' ability to adapt to evolving user interests in real-time systems.

RANK_REASON The cluster describes a new academic paper introducing a benchmark and dataset for evaluating LLMs.

Read on Hugging Face Daily Papers →

paper
other

AI-generated summary · Google Gemini · from 3 sources. How we write summaries →

New benchmark tests LLMs on dynamic user profiling from streaming content

COVERAGE [3]

arXiv cs.CL TIER_1 English(EN) · Sizhe Wang, Feiyu Duan, Juelin Wang, Liwen Zhang, Feiyu Duan · 2026-05-26 04:00

StreamProfileBench: A Benchmark for Fine-Grained User Profile Inference in Real-World Streaming Scenarios

arXiv:2605.25758v1 Announce Type: new Abstract: Large Language Models (LLMs) have reshaped user profiling, yet current evaluations mainly focus on static data snapshots. This paradigm overlooks the reality of personalized systems, where User-Generated Content (UGC) arrives contin…
Hugging Face Daily Papers TIER_1 English(EN) · 2026-05-25 12:10

StreamProfileBench: A Benchmark for Fine-Grained User Profile Inference in Real-World Streaming Scenarios

Large Language Models (LLMs) have reshaped user profiling, yet current evaluations mainly focus on static data snapshots. This paradigm overlooks the reality of personalized systems, where User-Generated Content (UGC) arrives continuously and fine-grained profile evolve rapidly. …
arXiv cs.CL TIER_1 English(EN) · Feiyu Duan · 2026-05-25 12:10

StreamProfileBench: A Benchmark for Fine-Grained User Profile Inference in Real-World Streaming Scenarios

Large Language Models (LLMs) have reshaped user profiling, yet current evaluations mainly focus on static data snapshots. This paradigm overlooks the reality of personalized systems, where User-Generated Content (UGC) arrives continuously and fine-grained profile evolve rapidly. …

COVERAGE [3]

StreamProfileBench: A Benchmark for Fine-Grained User Profile Inference in Real-World Streaming Scenarios

StreamProfileBench: A Benchmark for Fine-Grained User Profile Inference in Real-World Streaming Scenarios

StreamProfileBench: A Benchmark for Fine-Grained User Profile Inference in Real-World Streaming Scenarios

RELATED ENTITIES

RELATED TOPICS