The Mirrored Influence Hypothesis: Efficient Data Influence Estimation by Harnessing Forward Passes
Researchers have introduced the Mirrored Influence Hypothesis, which suggests that understanding training data's influence on model predictions can be inverted to assess how training on test data would alter predictions for training samples. This new approach, which involves calculating gradients for test samples and a forward pass for training points, offers significant efficiency gains over existing methods, especially when test datasets are much smaller than training datasets. The method has demonstrated applicability in areas such as data attribution for diffusion models, detecting data leakage and mislabeled data, and analyzing memorization and behavior in language models. AI
IMPACT Provides a more efficient method for understanding data influence, potentially improving model trustworthiness and aiding in tasks like data leakage detection.