heterogenous attention structures

New methods unveiled for interpreting transformer attention circuits

Two new research papers propose methods for interpreting the internal workings of transformer models, particularly focusing on their attention mechanisms. The first paper introduces a generic interpretation approach for…