Ego-METAS: Egocentric online Multimodal Energy-efficient Temporal Action Segmentation benchmark
Researchers have introduced Ego-METAS, a new benchmark designed for egocentric, multimodal, and energy-efficient temporal action segmentation. This benchmark utilizes over 100 hours of egocentric video data from three datasets, incorporating five sensor modalities including RGB, audio, gaze, IMU, and monochrome cameras. The task requires models to dynamically select which sensors to activate within strict energy budgets, addressing the under-explored area of energy-aware perception for embodied agents. Initial evaluations indicate that optimal sensor routing is highly dependent on the specific scenario, and current policy-learning methods struggle with continuous, untrimmed environments, though even simple dynamic fusion of modalities can be critical for balancing accuracy and energy constraints. AI
IMPACT Establishes a new standard for developing energy-conscious perception systems in embodied AI, crucial for real-world robotic applications.