Researchers have disproven a conjecture regarding the transition time bounds for separable logistic regression under gradient descent with large constant step sizes. Previously, it was believed that the transition time would be independent of the step size in dimensions greater than or equal to two. However, this paper demonstrates that for a fixed sample size and a sufficiently small margin, the transition time is dependent on the logarithm of the step size, specifically scaling with $(\log\eta)^{\min\{n-2,d-2\}}$. This finding was established by controlling the changes in the sample that most strongly influences the gradient and constructing matching hard instances. AI
IMPACT This research refines theoretical understanding of gradient descent dynamics, potentially impacting optimization algorithm design.
RANK_REASON The item is an academic paper detailing theoretical findings in machine learning. [lever_c_demoted from research: ic=1 ai=1.0]
- alphaXiv
- arXiv
- CatalyzeX
- DagsHub
- Edge of Stability
- gradient descent
- Hugging Face
- IArxiv
- logistic regression
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →