The Expense of Seeing: Attaining Trustworthy Multimodal Reasoning Within the Monolithic Paradigm
Researchers have developed Faithful-MR1, a new training framework designed to improve the faithfulness of multimodal reasoning in large language models. This framework addresses the challenge of accurately perceiving and utilizing visual information during reasoning by anchoring and reinforcing visual attention. Experiments show Faithful-MR1 outperforms existing baselines on Qwen2.5-VL-Instruct models with less training data. Separately, another paper critiques the trustworthiness of current Vision-Language Models, arguing they often rely on language priors rather than genuine visual understanding and proposing new metrics to evaluate this 'Expense of Seeing'. AI
IMPACT New research introduces methods to improve visual faithfulness in multimodal AI and critiques current evaluation practices, potentially guiding future model development.