New IG-Lens method precisely attributes token probability across transformer layers

Researchers have developed IG-Lens, a novel method for precisely attributing the probability of a predicted token to specific layers within decoder-only transformer models. Unlike existing tools that offer approximate or biased estimations, IG-Lens uses a telescoping application of Integrated Gradients to provide an exact additive decomposition in probability space. This approach accounts for the softmax nonlinearity, ensuring that the sum of attributions across layers precisely matches the total change in prediction probability. AI

IMPACT Provides a more accurate method for understanding internal model behavior, potentially aiding in debugging and interpretability.

RANK_REASON The cluster contains a research paper detailing a new method for analyzing transformer models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• decoder-only transformers
• IG-Lens
• Integrated Gradients
• Transformer Layers

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →