supported-Polar hidden updates

Neural networks can learn with flat training loss, study finds

Researchers have demonstrated that neural networks can continue to learn and improve their performance even when their training loss remains relatively flat. Using ReLU networks and specific update rules on the modular …