New reward model jointly assesses visual quality and action-following fidelity

Researchers have developed ACWM-RM, a novel reward model designed to improve action-conditioned world models (ACWMs) by jointly evaluating both visual quality (VQ) and action-following fidelity (AF) without requiring ground-truth videos. This model addresses limitations of existing rewards, which typically focus on only one dimension or struggle with generated content. To train ACWM-RM, a new dataset called AVM-Pref was created, containing over 500,000 preference pairs for complementary supervision. Evaluations on the AVM-Eval benchmark demonstrated ACWM-RM's superior performance over baseline rewards in reinforcement learning experiments. AI

IMPACT This new reward model could enhance the training of action-conditioned world models, leading to more realistic and accurate generated videos.

RANK_REASON The cluster contains a research paper detailing a new model and dataset. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• Action and Video Mismatch Benchmark
• Action and Video Mismatch Preference Dataset
• ACWM-RM
• arXiv
• AVM-Eval
• AVM-Pref
• Hugging Face

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →