Asymmetric Weight Reversion

New research reveals early-emergent robustness in neural networks is lost during training

Researchers have identified a phenomenon in deep neural networks where robustness to natural corruptions emerges early in the training process but is subsequently lost. They propose a framework called Early-Phase Stabil…