How Accurately Can a Gaussian Approximate Stochastic Approximation Iterates?
Researchers have developed a novel method to approximate the distribution of stochastic approximation (SA) iterates in finite time. The approach uses a sequence of Gaussians with recursively defined covariance to bound the pre-limit distributions. This work establishes explicit bounds on the Wasserstein-1 distance between the rescaled iterate and the Gaussian approximation, providing convergence rates for asymptotic normality and tail bounds on SA iterate errors. AI
IMPACT Provides a new theoretical framework for analyzing noisy iterative algorithms, potentially improving the understanding and development of machine learning optimization techniques.