Aleksandra Bakalova

Researchers Decompile Transformers into Interpretable RASP Programs

Researchers have developed a new method to extract interpretable algorithms from trained Transformer models. This technique involves re-parameterizing the Transformer into a RASP program and then using causal interventi…