New optimization methods achieve optimal sample complexity for complex nested functions

This paper introduces novel momentum-based methods for stochastic multi-level optimization, where the objective function is a nested composition of several non-convex functions. The proposed techniques utilize mini-batches to estimate function values at each level, which then inform the construction of momentum gradient estimators. The research establishes an optimal sample complexity of O(epsilon^-4) for finding an epsilon-stationary point, a significant improvement over previous assumptions. Additionally, a batch-free method is presented that achieves the same optimal rate without requiring problem-specific hyperparameter tuning. The effectiveness of these methods is demonstrated through applications in risk-averse portfolio optimization and hierarchical tilted empirical risk minimization. AI

IMPACT Introduces theoretical advancements in optimization that could underpin future AI model training techniques.

RANK_REASON The item is an academic paper detailing new theoretical methods and experimental validation in optimization. [lever_c_demoted from research: ic=1 ai=0.7]

Read on arXiv cs.LG →

• alphaXiv
• arXiv
• CatalyzeX Code Finder for Papers
• Connected Papers
• CORE Recommender
• DagsHub
• Gotit.pub
• hierarchical tilted empirical risk minimization
• Hugging Face
• Influence Flower
• Litmaps
• risk-averse portfolio optimization
• ScienceCast
• scite Smart Citations
• Stochastic Multilevel Compositional Optimization

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →