Look Again Before You Abstain:Budgeted Conformal Evidence Acquisition for Reliable Vision-Language Model
Researchers have developed a new method called Budgeted Conformal Evidence Acquisition (BCEA) to address hallucinations in large vision-language models (LVLMs). Traditional methods that require abstaining from predictions to maintain accuracy are highly inefficient, often abstaining on over 80% of claims. BCEA offers a more nuanced approach by allowing models to either answer, abstain, or acquire additional visual evidence within a compute budget, thereby restoring statistical guarantees and improving coverage. AI
IMPACT This research offers a more efficient way to ensure the accuracy of vision-language models by intelligently acquiring more data rather than simply abstaining from predictions.