Do Joint Audio-Video Generation Models Understand Physics?
Researchers have developed a new benchmark called AV-Phys Bench to evaluate the physical commonsense understanding of joint audio-video generation models. The benchmark tests models on their ability to generate consistent audio and video across steady states, event transitions, and environment transitions. While Seedance 2.0 showed the best performance, all tested models, including proprietary ones, struggled significantly with physically inconsistent prompts and dynamic scene changes, indicating that robust physical understanding remains a major challenge in this field. AI
IMPACT Highlights critical gaps in AI's ability to understand and generate physically consistent multimodal content, guiding future research.