On the Reliability of Cue Conflict and Beyond
Researchers have developed REFINED-BIAS, a new framework and dataset designed to more reliably diagnose shape and texture biases in neural networks. The existing cue-conflict benchmark has shown limitations in accurately measuring these biases due to issues with cue separation and evaluation scope. REFINED-BIAS addresses these by using explicit definitions for shape and texture, creating balanced cue pairs, and employing a ranking-based metric across the full label space for more accurate cross-model comparisons. AI
IMPACT Provides a more accurate method for evaluating and understanding shape and texture biases in AI models, leading to more reliable AI development.