Expectation Consistency Loss: Rethink Confidence Calibration under Covariate Shift
Researchers have developed a new method called Expectation Consistency Loss (ECL) to improve confidence calibration in classification models, particularly when dealing with covariate shifts. This approach rethinks calibration by establishing an "Expectation consistency condition," which suggests that shifts in data distribution do not inherently lead to uncalibrated confidence. ECL is an unsupervised domain adaptation loss that can be applied to various calibration types and is proven to have similar sample complexity to existing methods like Expected Calibration Error (ECE). The effectiveness of ECL has been validated on both simulated and real-world datasets. AI
IMPACT Enhances reliability of AI models in safety-critical applications by improving confidence calibration under changing data conditions.