New Affine-Invariant Cubic Newton scheme improves weak learner optimization

Researchers have developed a new optimization scheme called Affine-Invariant Cubic Newton (AICN) tailored for convex optimization with weak learners. This method improves upon existing Gradient-Regularized Newton (GRN) boosting algorithms by incorporating Hessian geometry, leading to a better theoretical convergence rate of \mathcal{O}(1/\overline{\Theta}_k^4 k^2) compared to GRN's \mathcal{O}(1/\gamma^{12}k^2). The AICN framework also naturally accommodates Newton coordinate descent as a special case and introduces an adaptive variant, weak-AdaAICN. AI

IMPACT Introduces theoretical improvements for optimization algorithms used in machine learning, potentially enhancing the performance and understanding of gradient boosting methods.

RANK_REASON Academic paper detailing a new optimization algorithm. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv stat.ML →

• Affine-Invariant Cubic Newton (AICN)
• Gradient-Regularized Newton (GRN)
• Newton's method
• Nikita Zozoulenko

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →