A new research paper explores finite-step gradient descent not as a simple optimization tool, but as a discrete dynamical system. The study analyzes how the training map's behavior, including phenomena like edge-of-stability and oscillations, is influenced by the learning rate. By examining simplified models of deep learning, the research suggests that the learning rate is a fundamental structural parameter that shapes the representations selected by gradient descent, rather than just a numerical stability constant. AI
IMPACT This research reframes gradient descent as a dynamical system, potentially leading to new insights into optimization and model training.
RANK_REASON The cluster contains a research paper submitted to arXiv detailing a new theoretical framework for understanding gradient descent.
- alphaXiv
- arXiv
- CatalyzeX Code Finder for Papers
- CORE Recommender
- cs.AI
- DagsHub
- deep learning
- Edge of Stability
- Gotit.pub
- gradient descent
- Hugging Face
- Influence Flower
- Ricker-type map
- ScienceCast
- The Map Behind the Flow: Finite-Step Gradient Descent as a Dynamical System
AI-generated summary · Google Gemini · from 2 sources. How we write summaries →
