New research details optimal approximation of smooth functions by ReLU neural networks

This paper explores the optimal approximation of smooth functions using ReLU neural networks, specifically addressing the uniform approximation of the Hölder ball $\mathcal H_1^s([0,1]^D)$. The research closes the gap between existing constructions and the information-theoretic efficiency limit by developing networks with width $O(N)$ and depth $O(L)$ that achieve an approximation error of order $(N^2L^2\log N)^{-s/D}$. This construction utilizes novel encoding and decoding mechanisms that leverage higher-order finite-difference bounds for local Taylor coefficients, leading to empirical risk minimizers that attain the minimax regression rate $n^{-2s/(2s+D)}$ without extra logarithmic factors. AI

IMPACT Provides theoretical underpinnings for improved function approximation in neural networks, potentially impacting model efficiency and performance.

RANK_REASON Academic paper detailing theoretical advancements in neural network approximation. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• [0,1]^d
• alphaXiv
• arXiv
• CatalyzeX
• DagsHub
• empirical risk minimizers
• Gotit.pub
• Hölder ball
• Hugging Face
• minimax regression rate
• rectifier
• ScienceCast

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →