Conformal Bayes under Label Shift: Post-Hoc Calibration vs. In-Training Adaptation
Researchers have introduced a new framework called Conformal Bayes that combines Bayesian posterior predictives with conformal calibration for more accurate prediction sets. The study explores two methods for handling label shift: post-hoc calibration, which adjusts predictions and thresholds without altering the model's core parameters, and in-training adaptation, which modifies the model's parameters directly to better suit the target domain. Experiments indicate that both approaches achieve valid coverage under unbiased training, while in-training adaptation offers improved efficiency by reducing interval width in optimization scenarios. AI
IMPACT Introduces a novel statistical framework for improving the reliability of AI predictions under data distribution changes.