RULER: Representation-Level Verification of Machine Unlearning
Researchers have developed RULER, a new set of metrics designed to verify machine unlearning at the representation level. Current methods only check output-level compliance, which can still leave residual information in a model's intermediate representations. RULER introduces two metrics, M2 and M4, to detect these residuals. Experiments showed that four out of five tested unlearning methods passed output-level evaluations but still contained significant residuals, particularly as the proportion of data to be unlearned increased. RULER also functions as a pre-unlearning diagnostic tool, identifying memorization issues in various data types. AI
IMPACT Introduces novel verification methods that could improve the robustness of machine unlearning techniques.