Challenges in the calibration of tree-based models for imbalanced classification
A new arXiv paper by Nathan Phelps details challenges in calibrating tree-based models for imbalanced classification tasks. The research highlights that analytical calibration methods, commonly used to adjust for majority class subsampling, can negatively impact prediction accuracy. Phelps demonstrates that these methods can lead to prevalence estimates that are dependent on factors like the number of predictors used in random forest splits and the sampling rate itself. The paper suggests that alternative calibration approaches, such as beta calibration, which can learn miscalibration patterns directly from the model, are more suitable for tree-based models trained on undersampled data. AI
IMPACT Highlights potential inaccuracies in common machine learning practices for imbalanced datasets, suggesting alternative calibration methods.