Researchers have identified a new capacity parameter for gradient boosting decision trees (GBDTs) called split-candidate scaling, which can lead to a phenomenon known as double descent. Unlike neural networks, GBDTs have lacked a clear single-axis capacity parameter. This new approach involves adjusting the number of split candidates, which refines the feature-quantization grid and expands the available paths for boosting updates. Experiments with XGBoost, LightGBM, and Catboost demonstrated that increasing split candidates can cause test error to peak and then decrease, a behavior not observed in random forests. AI
IMPACT Introduces a new theoretical framework for understanding GBDT capacity, potentially leading to improved model tuning and performance.
RANK_REASON The cluster describes a new research paper published on arXiv detailing a novel theoretical finding and empirical analysis of gradient boosting decision trees.
- arXiv
- Catboost
- double descent
- Gradient Boosting Decision Trees
- LightGBM
- random forest
- XGBoost
- split candidates
- tree-kernel diagnostic
AI-generated summary · Google Gemini · from 2 sources. How we write summaries →
