Right Regions, Wrong Labels: Semantic Label Flips in Segmentation under Correlation Shift
Researchers have identified a specific failure mode in semantic segmentation models, termed 'semantic label flips,' where models correctly identify object boundaries but assign incorrect semantic labels to foreground pixels. This issue is exacerbated by correlation shifts between training and testing data, particularly when non-causal features are strongly tied to labels. The study proposes a new metric, 'Flip,' to quantify these within-object label swaps and an entropy-based 'flip-risk' score to detect such cases during inference. AI
IMPACT Highlights a critical robustness issue in segmentation models, potentially impacting real-world applications and guiding future research towards more reliable AI systems.