Researchers have developed MINT (Minting IN-the-Wild Trajectories), a novel foundation model capable of estimating both camera and hand motion in world coordinates directly from egocentric RGB video. Unlike previous systems that process these tasks separately, MINT jointly predicts camera trajectory, hand states, and hand presence from a shared video representation. To overcome the scarcity of world-space annotations, an open-source labeling pipeline called EGOPIPELINE was created to generate large-scale pseudo-labels for training. MINT demonstrates significant improvements in accuracy and speed over existing methods and is released with its code, training data, and labeling pipeline. AI
IMPACT Enables more accurate and efficient activity understanding in robotics and AR by jointly modeling camera and hand motion.
RANK_REASON This is a research paper detailing a new model and associated open-source tools for computer vision tasks. [lever_c_demoted from research: ic=1 ai=1.0]
- alphaXiv
- arXiv
- CatalyzeX
- Connected Papers
- CORE Recommender
- DagsHub
- EGOPIPELINE
- Gotit.pub
- Hugging Face
- Influence Flower
- Litmaps
- ScienceCast
- scite Smart Citations
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
