PulseAugur
EN
LIVE 06:41:06

SGD Provably Learns Spurious Features First in Neural Networks

By PulseAugur Editorial · [2 sources] ·

A new theoretical study published on arXiv investigates how Stochastic Gradient Descent (SGD) learns spurious features in two-layer ReLU neural networks. The research demonstrates that SGD prioritizes and learns these spurious correlations exponentially fast, even before learning the actual signal. The study's analysis reveals that the optimization dynamics can couple the spurious and signal features, potentially inhibiting the learning of the true signal, especially when the spurious correlation is strong. AI

IMPACT This research provides a theoretical understanding of how neural networks can learn spurious correlations, potentially informing the development of more robust training algorithms.

RANK_REASON The cluster contains a research paper detailing theoretical findings on neural network training dynamics.

Read on arXiv stat.ML →

AI-generated summary · Google Gemini · from 2 sources. How we write summaries →

SGD Provably Learns Spurious Features First in Neural Networks

COVERAGE [2]

  1. arXiv stat.ML TIER_1 English(EN) · Tyler LaBonte, Vidya Muthukumar ·

    SGD Provably Prioritizes a Shortcut Spurious Feature in the XOR Model

    arXiv:2606.30444v1 Announce Type: new Abstract: Neural networks are known to be susceptible to over-reliance on spurious correlations. However, the precise mechanism by which models exploit shortcut features is not fully understood, and algorithms to mitigate this behavior rely o…

  2. arXiv stat.ML TIER_1 English(EN) · Vidya Muthukumar ·

    SGD Provably Prioritizes a Shortcut Spurious Feature in the XOR Model

    Neural networks are known to be susceptible to over-reliance on spurious correlations. However, the precise mechanism by which models exploit shortcut features is not fully understood, and algorithms to mitigate this behavior rely on as yet unjustified assumptions about the learn…

RELATED ENTITIES

RELATED TOPICS