English(EN)Ego2World: Compiling Egocentric Cooking Videos into Executable Worlds for Belief-State Planning
新基准评估AI的主观视角视频操控和工具使用技能
作者PulseAugur 编辑部·[7 个来源]·
研究人员推出了Ego2Act,这是一个旨在评估视频生成模型在主观视角视频中执行目标导向操作能力的新基准。该基准包含跨越110个现实世界任务的2,640个视频,突显了当前模型在多步物理推理方面遇到的困难,常常会跳过或不完整地执行动作。为解决在现实世界主观视角视频中对工具使用理解的需求,已开发出EgoTools,它包含一个大型数据集和一个诊断基准。虽然Gemini-3.1 Pro等模型显示出潜力,但它们在将工具使用与视觉证据联系起来方面仍有不足,表明该领域需要进一步发展。
AI
arXiv:2605.13335v2 Announce Type: replace Abstract: Embodied agents in household environments must plan under partial observation: they need to remember objects, track state changes, and recover when actions fail. Existing benchmarks only partially test this ability. Egocentric v…
arXiv:2610.02715v1 Announce Type: new Abstract: Egocentric videos capture how people carry out everyday activities, yet testing an agent requires evaluating the consequences of actions it chooses itself. We introduce Ego2World, a benchmark that turns annotated cooking activities …
Video generation models are increasingly being explored as world simulators for embodied planning and learning. To do so effectively, these models must not only generate visually appealing frames, but also predict how environments dynamically evolve when executing goal-directed a…
Real-world embodied tasks, from everyday activities to professional procedures, require agents to act under physical constraints while tracking evolving object and task states. Tool use sits at the heart of such tasks, as many everyday and professional activities are tool-mediate…
arXiv cs.CV
TIER_1English(EN)·Yifan Shen, Jiateng Liu, Xinzhuo Li, Yuanzhe Liu, Bingxuan Li, Houze Yang, Wenqi Jia, Yijiang Li, Tianjiao Yu, James Matthew Rehg, Xu Cao, Ismini Lourentzou·
arXiv:2603.20169v2 Announce Type: replace Abstract: Generative world models have shown promise for simulating dynamic environments, yet egocentric video remains challenging due to rapid viewpoint changes, frequent hand-object interactions, and goal-directed procedures whose evolu…
arXiv cs.CV
TIER_1English(EN)·Patrick Amadeus Irawan, Iskandar Muda Rizky Parlambang, Rava Maulana, Qinrong Cui, Erland Hilman Fuadi, Zayd M. K. Zuhri, Nanda Ryaas Absar, Ahmed Elshabrawy, Wilfried Ariel Mulyawan, Shoubin Yu, Yue Zhang, Mohit Bansal, Alham Fikri Aji·
arXiv:2610.01092v1 Announce Type: new Abstract: Video generation models are increasingly being explored as world simulators for embodied planning and learning. To do so effectively, these models must not only generate visually appealing frames, but also predict how environments d…
arXiv:2609.39378v1 Announce Type: new Abstract: Real-world embodied tasks, from everyday activities to professional procedures, require agents to act under physical constraints while tracking evolving object and task states. Tool use sits at the heart of such tasks, as many every…