Critical Organization of Deep Neural Networks, and p-Adic Statistical Field Theories
Researchers have rigorously studied the thermodynamic limit of deep neural networks (DNNs) and recurrent neural networks (RNNs), focusing on sigmoid activation functions. They demonstrated that in a specific parameter region, these networks exhibit a unique state that transitions to infinitely many states outside this region, a phenomenon termed critical organization. The study also utilizes p-adic integers to represent hierarchical structures within these networks, connecting critical organization to p-adic tree-like structures and analyzing a toy model of a hierarchical edge detector. AI
IMPACT Provides theoretical insights into the behavior and structure of deep learning models.