Wang et al. (2023)

New Unpack method deciphers transformer component interactions

Researchers have developed a new method called Unpack to analyze the internal workings of transformer models. This technique uses backward recursion to trace how different components, like attention and MLP layers, cont…