When Accuracy Is Not Enough: Uncertainty Collapse between Noisy Label Learning and Out-of-Distribution Detection
Researchers have developed a new method called Standardized Loss Aggregation (SLA) to detect noisy labels in large datasets, particularly in medical imaging. SLA quantifies label reliability by analyzing standardized losses from cross-validation runs, offering a more continuous and informative measure than simple hard-counting methods. Experiments show SLA is more effective and faster at identifying ambiguous or mislabeled samples, which can help improve dataset quality for classification tasks. Another study highlights a problem called "uncertainty collapse" where models trained on noisy labels achieve high accuracy but fail to reliably distinguish out-of-distribution data from misclassified in-distribution data. AI
IMPACT New techniques for handling noisy labels can improve the reliability and robustness of AI models, especially in critical domains like medical imaging.