New study reveals double descent in Gradient Boosting Decision Trees

Researchers have identified a new capacity parameter for gradient boosting decision trees (GBDTs) called split-candidate scaling, which influences how the model learns. By analyzing the number of split candidates, they observed a double descent phenomenon, where test error initially increases before decreasing as the budget for split candidates grows. This behavior was noted across popular GBDT implementations like XGBoost, LightGBM, and CatBoost, but not in random forests, suggesting a unique characteristic of boosting dynamics. AI

IMPACT This research provides a new theoretical lens for understanding the behavior of gradient boosting decision trees, potentially leading to improved model tuning and performance.

RANK_REASON Academic paper on a theoretical aspect of machine learning models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• arXiv
• Catboost
• double descent
• Gradient Boosting Decision Trees
• LightGBM
• random forest
• XGBoost

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →