Deep Heaviside Networks: Expressivity Enhanced by Skip Connections and Linear Activations

A new paper published on arXiv explores the expressivity of deep Heaviside networks (DHNs). The research demonstrates that while DHNs have inherent limitations in their expressive power, these can be mitigated by incorporating skip connections or neurons with linear activation. The study provides theoretical bounds on the Vapnik-Chervonenkis dimensions and approximation rates for these enhanced network architectures, with a specific application to deriving statistical convergence rates for DHN fits in nonparametric regression models. AI

IMPACT This research contributes to the theoretical understanding of neural network capabilities, potentially informing the design of more efficient models.

RANK_REASON The cluster contains an academic paper detailing theoretical advancements in neural network expressivity. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv stat.ML →

• arXiv
• Heaviside networks
• Insung Kong
• Vapnik-Chervonenkis dimensions

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →