A new research paper explores population loss in shallow ReLU networks, specifically focusing on networks with bias. The study presents a formula for population loss in the student-teacher kernel model, extending prior work and utilizing Owen's T-function. It demonstrates that adding bias to these networks consistently decreases the loss and shows that various families of spurious minima found in unbiased networks also extend to biased ones, with only mild changes to the landscape geometry. AI
IMPACT Provides theoretical insights into the behavior of ReLU networks, potentially informing future model design and optimization.
RANK_REASON Academic paper detailing mathematical findings on neural network properties. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
