Dana et al.

New research details incremental learning in overparameterized ReLU networks

Researchers have published a paper detailing the incremental learning process in mildly overparameterized ReLU networks trained on orthogonal data. The study proves that as initialization scale approaches zero, the grad…