Boursier et al.

ReLU Activation's Impact on Gradient Descent Bias in Neural Networks Detailed

A new research paper explores how the ReLU activation function influences the implicit bias of gradient descent in high-dimensional neural network regression. The study, using a novel primal-dual analysis, demonstrates …