A new paper demonstrates that deep ReLU networks can fail to achieve the best approximation property for continuous target functions. Researchers constructed a specific continuous function on a 2D domain where the minimum L^2-approximation error could not be reached by networks with two hidden layers and a (2,2,1,1) architecture. This finding contrasts with shallow neural networks, where ReLU activations typically ensure global minima for regression tasks, and is attributed to the bending of breaklines in deeper layers. AI
IMPACT Highlights a theoretical limitation in deep ReLU networks, potentially impacting their use in certain approximation tasks.
RANK_REASON Academic paper published on arXiv detailing a theoretical finding about neural network properties. [lever_c_demoted from research: ic=1 ai=1.0]
- architecture (2,2,1,1)
- arXiv
- continuous target functions
- L^2-approximation error
- regression tasks
- ReLU networks
- two hidden layers
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
