New framework reveals Vision Transformers are less reliant on texture than CNNs

A new research paper proposes a semantically matched evaluation framework to better understand how visual recognition models, like CNNs and Vision Transformers (ViTs), rely on different features such as shape and texture. Previous studies, which used artificial cue conflicts, suggested CNNs were heavily texture-biased. However, this new framework demonstrates that ImageNet-trained CNNs show greater degradation when texture is suppressed compared to shape, indicating a stronger reliance on texture. The research also found that ViTs maintain higher accuracy and show less degradation under both shape and texture suppression compared to CNNs, suggesting their representations are more aligned with the human visual cortex. AI

IMPACT Introduces a more robust method for evaluating AI model feature reliance, potentially leading to better interpretability and development of more human-aligned AI.

RANK_REASON Research paper introducing a new evaluation framework for AI models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• CNNs
• ImageNet
• Vision Transformers

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →