Researchers have explored the geometric properties and closure of function spaces realized by ReLU neural networks. They established a general result on limits of continuous piecewise-linear functions, showing that the minimum number of convex sets on which a function is affine is lower semicontinuous under pointwise convergence. This work characterizes functions realizable by shallow ReLU networks and proves that fixed-width shallow scalar ReLU realization spaces are closed in the weighted supremum norm. However, they also constructed a sequence of networks whose limit is not realizable by the same architecture, disproving a conjecture by Petersen, Raslan, and Voigtlaender regarding non-closedness. AI
IMPACT Provides theoretical tools for understanding the boundaries and degenerations of ReLU realization spaces, potentially influencing future neural network architecture design.
RANK_REASON The cluster contains an academic paper detailing theoretical research on neural network function spaces. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
