Counter languages

Study shows stack representations are causally necessary for transformer language models

Researchers have published a paper demonstrating the causal necessity of stack representations in transformer models for processing counter languages. By training linear probes to predict stack depth and then ablating t…