Deep Residual Networks Learn Geodesic Curves in Wasserstein Space

A new arXiv paper proposes that deep residual networks (ResNets) learn the geodesic curve within Wasserstein space during training. The research models ResNet forward propagation using continuity equations, suggesting that ResNets with L2 regularization approximate this geodesic curve more effectively than plain networks. This improved approximation is posited as a reason for ResNets' better optimization and generalization capabilities. AI

IMPACT This research offers theoretical insights into the optimization and generalization of ResNets, potentially informing future network architectures.

RANK_REASON The cluster contains a single academic paper detailing a theoretical finding about deep neural networks. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv stat.ML →

• arXiv
• Continuity equations
• geodesic curve
• L2 regularization
• plain networks
• Shihua Zhang
• Wasserstein space

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →