New analysis details gradient descent performance in logistic regression

This paper analyzes the finite-sample performance of gradient descent in logistic regression with Gaussian design. The authors establish that gradient descent can achieve linear convergence to a small neighborhood of the true parameter, with an $\ell_2$ error of order $O(\sqrt{\|\theta^*\|_2^5d/n})$ under a small stepsize. They also demonstrate a faster local linear convergence with a larger stepsize. A key technical contribution is showing that the gradient of the logistic loss satisfies an approximate invertibility condition, which is achieved through uniform control of gradient deviations and delicate analysis of population Hessian eigenvalues. The research also proposes a novel efficient estimator that achieves a sharper rate in high dimensions, suggesting that $O(\sqrt{\|\theta^*\|_2d/n})$ is the tight estimation error rate in many regimes. AI

IMPACT Provides theoretical guarantees for optimization algorithms used in machine learning models.

RANK_REASON Academic paper detailing theoretical analysis of an algorithm. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv stat.ML →

• Gaussian Design
• gradient descent
• logistic regression
• Matsumoto
• Mazumdar

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →