New research optimizes train-test splits for better AutoML evaluation

A new research paper explores methods for splitting datasets in machine learning to improve model evaluation, particularly for datasets with class imbalance or natural clustering. The study compares five established splitting strategies, including random splitting, stratified sampling, Kennard-Stone, Duplex, and SPXY, assessing their statistical similarity using chi-square, Kolmogorov-Smirnov, and Maximum Mean Discrepancy (MMD) tests. Results indicate that geometry-based methods can introduce instability, while a proposed "Optimised-Distribution" method aims to optimize similarity as an explicit objective, achieving a high mean MMD similarity score. AI

IMPACT Improves the reliability of AutoML model evaluation by addressing issues with dataset splitting.

RANK_REASON Academic paper detailing new methods for machine learning dataset splitting. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• chi-squared distribution
• Duplex
• Kennard-Stone
• Kolmogorov–Smirnov test
• Maximum Mean Discrepancy
• Optimised-Distribution
• SPXY
• University of California, Irvine

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →