Representation Costs in Data Science: Foundations and the Quasi-Banach Spaces of Deep Neural Networks
A new research paper introduces a unified framework for analyzing the representation costs of parametric data-fitting methods. This framework reveals the induced function spaces for various models, including kernel methods, wavelets, and shallow neural networks, as special cases. For deep neural networks with ReLU activations, the paper demonstrates that their native spaces are quasi-Banach spaces where the inductive bias cannot be captured by norms for depths greater than two. AI
IMPACT This research provides a theoretical foundation for understanding the inductive biases of deep neural networks, potentially guiding future model design.
- Variation Spaces
- arXiv
- Relu Networks
- Reproducing Kernel Hilbert Spaces
- Shallow Neural Networks for mmWave Radar Based Recognition of Vulnerable Road Users
- Quasi-Banach Spaces
- Parametric models for incomplete continuous and categorical longitudinal data
- Representer Theorems
- kernel method
- function spaces
- wavelets
- Deep Neural Networks
- Besov Spaces
- DagsHub
- Hugging Face
- CatalyzeX Code Finder for Papers
- alphaXiv
- ScienceCast
- Gotit.pub