Assign and Add: A Mechanistic Study of Compositional Arithmetic
Researchers have published a study on compositional arithmetic in transformers, exploring how these models generalize to unseen combinations of variables and numbers. The study, titled "Assign and Add: A Mechanistic Study of Compositional Arithmetic," analyzes a controlled setting involving variable assignment and modular addition. Findings suggest that compositional generalization can naturally arise from the internal mechanisms of transformers, with training dynamics showing distinct learning phases. AI
IMPACT Provides theoretical and empirical insights into how transformers achieve compositional generalization, potentially informing future model architectures.