Sobolev Approximation by Fixed-Size Neural Networks with Arbitrary Accuracy
Researchers have introduced new activation functions, the Elementary Universal Activation Function (EUAF) and Differentiable Universal Activation Functions (DUAF), designed to enable fixed-size neural networks to achieve arbitrary-accuracy Sobolev approximation. The study demonstrates that functions within $W^{s, ext{inf}}((a,b)^d)$ can be approximated with arbitrary accuracy in the $W^{s-1, ext{inf}}$-norm using networks with these novel activations. Explicit bounds for network width and depth are provided, and sigmoidal variants of DUAF are also explored. AI
IMPACT Introduces novel activation functions that could enhance the approximation capabilities of fixed-size neural networks.