Learning dynamics reshape internal representations in neural networks and brains

A new paper published on arXiv details how learning processes reshape power-law anisotropy within the internal representations of neural networks. Researchers analyzed a two-layer linear neural network and found that the local power-law exponent evolves non-monotonically during training, potentially entering up to four distinct asymptotic regimes. This dynamic interaction between input statistics and task structure appears to be a general mechanism for generating power-law internal representations, a phenomenon observed in both artificial language models and biological systems like the mouse cerebral cortex. AI

IMPACT Provides insight into the fundamental mechanisms driving representation learning in AI models.

RANK_REASON The cluster contains a research paper detailing findings on neural network internal representations. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv stat.ML →

• arXiv
• CatalyzeX Code Finder for Papers
• CORE Recommender
• DagsHub
• Gotit.pub
• Hugging Face
• IArxiv Recommender
• Influence Flower
• Language Models
• mouse cerebral cortex
• ScienceCast
• two-layer linear neural network

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →