Mitigating Manifold Departure: Uncertainty-Aware Subspace Rectification for Trustworthy MLLM Decoding
Researchers have developed a new training-free decoding method called Manifold-Guided Adaptive Projection (MGAP) to combat hallucinations in Multimodal Large Language Models (MLLMs). This method addresses the issue where models generate objects inconsistent with visual inputs, often due to an over-reliance on language priors. MGAP works by identifying and adaptively attenuating the problematic language prior components within a constructed language-prior subspace, thereby preserving the essential semantic structure of the model's representations. Experiments on POPE and CHAIR benchmarks demonstrate that MGAP effectively suppresses hallucinations while maintaining coherence, outperforming existing decoding baselines. AI
IMPACT Mitigates hallucinations in MLLMs, potentially improving their reliability for multimodal tasks.