New Multiscale Single-Index Model for Hierarchical Feature Learning

Researchers have introduced the Multiscale Single-Index Model (MSIM), a stylized framework designed to study hierarchical feature learning with scale separation. This model analyzes how deep architectures learn representations across different scales by having each layer extract a shared single-index feature. The study details how MSIM relates to the Tensor PCA model and uses Edgeworth expansions for a fine-grained analysis of Wiener chaos, revealing structures that enable efficient spectral recovery and analysis of backpropagation methods. The findings suggest that online SGD can achieve near-perfect recovery with a sample complexity comparable to linear models. AI

IMPACT Introduces a new theoretical model for understanding hierarchical feature learning in deep architectures.

RANK_REASON The cluster contains an academic paper detailing a new model for hierarchical feature learning. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• montanari2014statistical
• Multiscale Single-Index Model
• oymak2021learning
• SGD
• Single-minded family bHLH transcription factor 2
• Tensor PCA

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →