New CoFire method improves video understanding models by preserving initial answer reliability

Researchers have developed a new reinforcement learning method called CoFire to improve video understanding models. This method addresses the issue of "confidence erosion," where the learning process can degrade the initial answer's reliability. CoFire ensures the initial answer remains dependable before reflection, leading to a 4.1-point average improvement in initial-answer accuracy and a 3.7-point gain in final accuracy compared to standard outcome-reward RL. The approach achieved 67.0% mean accuracy on benchmark datasets like Video-MME, MLVU, and LongVideoBench, outperforming a comparable 7B method by 1.7 points. AI

IMPACT Enhances reliability in video understanding models, potentially improving performance in applications requiring accurate moment identification.

RANK_REASON Research paper detailing a new method for video understanding models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• arXiv
• CoFire
• Hugging Face
• LongVideoBench
• MLVU
• reinforcement learning
• Video-MME

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →