Gradient Descent on Logistic Regression: Equal Norms and Convergence

A new research paper explores the behavior of gradient descent on logistic regression models, particularly when data points are normalized to have equal norms. The study proves that for linearly separable data, gradient descent iterates converge to the maximum-margin separator irrespective of step size. However, for non-separable data, cycles can emerge even with step sizes below the stability threshold. The paper demonstrates that while equal-norm datasets can still lead to cycles in higher dimensions, global convergence is achieved for the population loss when examples are uniformly distributed on a sphere. Additionally, in one dimension, an annulus condition is shown to ensure global convergence below the stability threshold, suggesting potential avenues for similar conditions in higher dimensions. AI

IMPACT Provides theoretical insights into the convergence properties of gradient descent for logistic regression, potentially informing the development of more stable and efficient training algorithms.

RANK_REASON This is a research paper published on arXiv detailing theoretical findings about gradient descent algorithms. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• alphaXiv
• arXiv
• CatalyzeX
• DagsHub
• Gotit.pub
• gradient descent
• Hugging Face
• IArxiv
• logistic regression model
• ScienceCast
• Si Yi Meng

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →