New SoftMCC framework offers threshold-free model selection for imbalanced classification

Researchers have introduced SoftMCC, a novel post-training framework designed to improve model selection for imbalanced binary classification tasks. Traditional methods often rely on the Matthews correlation coefficient (MCC), but its threshold-dependent nature can lead to unstable validation rankings. SoftMCC addresses this by using probability-valued confusion counts and a tie-aware selection protocol, aiming for threshold-free evaluation. Experiments across multiple settings indicate that SoftMCC offers superior stability and ranking compared to other metrics like AUPRC and [email protected], though its direct utility advantage in selected models was not significant. AI

IMPACT Introduces a new metric for evaluating machine learning models in imbalanced classification scenarios.

RANK_REASON The item is an academic paper introducing a new method for model selection in machine learning. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• AUPRC
• Brier skill score
• F1@best
• Friedman test
• Holm correction
• Kendall's W
• Matthews correlation coefficient
• [email protected]
• Nemenyi analysis
• Pearson
• SoftMCC
• Spearman

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →