Archetypal SAE stability shown to be initialization artifact

Researchers have found that the claimed stability of Archetypal Sparse Autoencoders (SAEs) is an artifact of their initialization and metric design, rather than an inherent property. By removing the deterministic initialization used in previous studies, the researchers demonstrated that the archetypal constraint offers no significant stabilization advantage. The study also highlights a preprocessing-dependent geometry issue that complicates the interpretation of endpoint stability metrics, suggesting that claims of feature stability require more rigorous trajectory diagnostics and initialization ablations. AI

IMPACT Challenges assumptions about feature stability in interpretability methods, potentially impacting how researchers evaluate and trust SAEs.

RANK_REASON Academic paper analyzing a specific technique within mechanistic interpretability. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• Archetypal SAEs
• Fel et al.
• NLP

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →