New SERUM framework extracts user behavior models from screen video

Researchers have developed SERUM, a novel framework for extracting and refining finite-state behavioral models from unstructured screen activity using hierarchical vision-language model annotation. This multi-pass system alternates between activity-recognition and intent-inference to improve label accuracy and reduce hallucination. SERUM has demonstrated its ability to create interpretable process models from egocentric screen videos without manual annotation, showing promise for scalable user modeling and behavioral understanding. AI

IMPACT Enables scalable user modeling and behavioral understanding from unstructured screen data.

RANK_REASON The item is a research paper detailing a new framework and methodology for user modeling. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• alphaXiv
• CatalyzeX
• CORE Recommender
• DagsHub
• Gotit.pub
• Hugging Face
• Influence Flower
• Markov models
• optimization
• ScienceCast
• vision-language model

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →