Researchers have introduced PersonaCUABench, a new benchmark designed to evaluate computer-use agents (CUAs) based on the diverse personas of users. Unlike traditional benchmarks that use fixed task specifications, PersonaCUABench simulates user interactions by having a persona agent adopt one of 44 distinct personas to rephrase instructions and ask clarifying questions. This approach revealed a significant performance drop, with mean task completion falling by 36% and user satisfaction decreasing from 84.1% to 62.7% when agents interacted with persona-driven requests compared to original instructions. The findings highlight the need for CUAs to be evaluated not just on task completion but also on their ability to understand and meet user-specific requirements and constraints. AI
IMPACT Highlights the need for AI agents to better understand and adapt to individual user needs and communication styles.
RANK_REASON The item is an academic paper introducing a new benchmark for evaluating AI agents. [lever_c_demoted from research: ic=1 ai=1.0]
- alphaXiv
- arXiv
- CatalyzeX
- Computer Use Agents
- CUAS
- DagsHub
- Gotit.pub
- Hugging Face
- Persona Agent
- PersonaCUABench
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
