Patrick Lutz

New method deciphers Transformer in-context classification dynamics

Researchers have developed a method to interpret how Transformer models perform in-context classification. By enforcing specific symmetries in the model's weights, they were able to identify an emergent, layer-wise upda…