Brief

Researchers have revisited model stitching, a technique that connects early layers of one AI model to later layers of another, to explore its applicability to Vision Foundation Models (VFMs). Their study found that training the connecting 'stitch' layer is crucial for maintaining accuracy, especially at shallower connection points. By using a feature-matching loss at the target model's penultimate layer, they demonstrated that heterogeneous VFMs can be reliably stitched together for various vision tasks, sometimes even surpassing the performance of the individual models. AI

Researchers have introduced AVA-Bench, a new benchmark designed to systematically evaluate vision foundation models (VFMs). This benchmark disentangles 14 foundational visual abilities, such as localization and spatial understanding, to pinpoint specific VFM weaknesses. AVA-Bench aims to move VFM selection from guesswork to principled engineering by providing a more transparent and comprehensive evaluation. The study also found that using a smaller LLM for evaluation can significantly reduce computational costs. AI