Immuno-VLM: Immunizing Large Vision-Language Models via Generative Semantic Antibodies for Open-World Trustworthiness
Researchers have introduced Immuno-VLM, a novel framework designed to enhance the trustworthiness of large vision-language models in open-world scenarios. This bio-inspired approach utilizes generative semantic antibodies, created by LLMs, to represent textual descriptions of potential outliers. By actively defining boundaries for known categories, Immuno-VLM aims to mitigate the 'Hubris of Semantics' where models confidently misclassify unknown anomalies. Experiments demonstrate that Immuno-VLM sets a new state-of-the-art in open-world trustworthiness. AI
IMPACT Enhances the reliability of vision-language models in real-world applications by reducing misclassifications of unknown data.