New theory explains cross-validation variance, favors K-fold

A new paper published on arXiv introduces a theoretical framework for understanding the variance in cross-validation estimates of predictive performance. The research establishes an exact finite-sample identity for this variance, showing it decomposes into sampling variance and variance due to random fold splitting. The findings indicate that K-fold cross-validation is variance-optimal at first order, while Monte Carlo and bootstrap methods are not. The paper also suggests that for certain metrics like mean squared error and AUC, specific schemes like leave-one-out or leave-pair-out offer the smallest variance. AI

IMPACT Provides theoretical grounding for selecting optimal cross-validation designs, potentially improving model evaluation robustness.

RANK_REASON Academic paper on statistical methods for machine learning. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• arXiv
• bootstrap cross-validation
• Cross-Validation Variance
• K-fold cross-validation
• leave-one-out cross validation
• leave-pair-out
• mean squared error
• Nathaniel O'Connell

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →