NEWTON: Agentic Planning for Physically Grounded Video Generation
Researchers have developed new methods for improving procedural planning and video generation by grounding them in instructional content and physical principles. One approach, RECIPE, uses reinforcement learning with a grounding quality reward to train models on large, noisy instructional video corpora, enhancing their ability to generate step-by-step plans. Another system, NEWTON, frames video generation as an agentic task, orchestrating various physics-aware tools and using a verifier for iterative re-planning to improve physical commonsense in generated videos. AI
IMPACT These methods could lead to more capable AI assistants that can understand and generate complex procedural tasks and physically realistic videos.