Classical ML models show power-law scaling on tabular data

A new study published on arXiv benchmarks classical machine learning models on tabular data, revealing that power laws accurately describe learning curves across various datasets and model families. The research found that tree ensembles, particularly Boosting and Random Forest, perform best on classification tasks with full data. Notably, exponents within model families showed approximate shared characteristics, suggesting a degree of predictive compressibility, though not universal applicability. The study also highlighted significant variance in implementation results even with fixed random states, indicating the impact of unconstrained protocol elements like preprocessing and missing value handling. AI

IMPACT Provides insights into the scaling behavior of classical ML models, useful for optimizing training and understanding performance limits on tabular data.

RANK_REASON The item is an academic paper detailing benchmark study results. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv stat.ML →

• arXiv
• Boosting
• lasso
• Linear
• logistics
• Random Forest
• ridge
• support vector machine

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →