From Correlation to Cause: A Five-Stage Methodology for Feature Analysis in Transformer Language Models
Researchers have developed a five-stage methodology for causal feature analysis in transformer language models, demonstrating its application on GPT-2 small for the Indirect Object Identification task. The method uses activation patching to identify key circuits and a sparse autoencoder to recover selective features, finding these features to be partially causal. Robustness testing revealed a gap between detection and causal robustness, while a cost-based deployment evaluation showed significant savings for an optimal monitor configuration. AI
IMPACT Provides a structured approach to understanding and potentially improving the interpretability and reliability of transformer models.