Convex Basins in Single-Index Model Loss Landscapes: Applications to Robust Recovery under Strong Adversarial Corruption
Researchers have developed a novel algorithm for robustly learning Gaussian Single Index Models (SIMs) even when faced with heavy-tailed noise and adversarial corruption. This new method provides the first robust recovery guarantees for a wide range of nonlinear SIMs, including those with non-monotonic link functions like GeLU and Swish, which are common in modern neural architectures. The algorithm establishes a dimension-independent convex basin around the true parameters, allowing for efficient recovery via spectral initialization and subsequent robust gradient descent, achieving an estimation error of O(σ√ε) with near-linear time complexity. AI
IMPACT Enhances robustness in AI models against adversarial attacks and noisy data, enabling wider application of nonlinear architectures.