Researchers have developed Omni-Woodpecker (OWP), a novel framework designed to mitigate cross-modal hallucinations in Omni LLMs (OLLMs). Unlike previous methods that focus on output correction, OWP operates by rebalancing internal representations. It identifies conflicts between modalities, separates grounding evidence from competing priors within hidden states, and then adaptively adjusts these representations to reduce misleading information before generating output. Experiments show OWP significantly improves performance on benchmarks like CMM and AVHBench when applied to models such as Qwen2.5-Omni-7B and VideoLLaMA2-AV, demonstrating its effectiveness and generalization capabilities. AI
IMPACT Introduces a new method for improving the reliability of multimodal LLMs by addressing hallucinations.
RANK_REASON The cluster contains an academic paper detailing a new method for LLMs. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
