MolmoMotion: Language-guided 3D motion forecasting
Allen AI has introduced MolmoMotion, a novel model designed for language-guided 3D motion forecasting. This model predicts the future 3D trajectories of points on an object based on an initial video frame and a textual description of the intended action. MolmoMotion aims to advance applications in robotics planning and trajectory-conditioned video generation by providing a more useful forward-looking motion prediction capability compared to existing retrospective methods. The release includes the model weights, a large dataset named MolmoMotion-1M, and a benchmark called PointMotionBench to evaluate motion forecasting accuracy. AI
IMPACT Enables more sophisticated robotics and video generation by predicting future object motion from language instructions.