How Optimality Structures Sparse Dictionaries: A Theory for Understanding SAE Representations
A new research paper explores the theoretical underpinnings of Sparse Autoencoders (SAEs), a technique used to interpret complex neural network representations. The study proposes a framework to understand what SAEs extract and how scientific conclusions can be drawn from them. By extending local optimality analyses, the research derives constraints that explain observed SAE behaviors like hierarchical splitting and the structure of residuals, aiming to inform the design of future models. AI
IMPACT Provides a theoretical framework for understanding and improving interpretable AI techniques like SAEs.