EgoExoMem: Cross-View Memory Reasoning over Synchronized Egocentric and Exocentric Videos
Researchers have introduced new benchmarks and synthetic data generation methods to improve the performance of large multimodal models (LMMs) on egocentric video data. The EgoBabyVLM benchmark focuses on language grounding from naturalistic, weakly-aligned egocentric video, highlighting current LMMs' limitations in this domain. Similarly, EgoExoMem addresses cross-view memory reasoning using synchronized egocentric and exocentric videos, revealing that existing models struggle to achieve high accuracy. To overcome data collection challenges, EgoInteract offers a controllable simulator for generating synthetic egocentric videos with dense annotations, demonstrating improved model performance on real-world benchmarks. AI
IMPACT Advances in egocentric video understanding could enable more sophisticated embodied AI agents and human-computer interaction systems.