Back to the Feature: Explaining Video Classifiers with Video Counterfactual Explanations
Researchers have developed a new framework called Back To The Feature (BTTF) to generate counterfactual explanations for video classifiers. Unlike previous methods focused on images, BTTF addresses the unique challenges of video explanations, ensuring they are plausible, temporally coherent, and exhibit smooth motion. The framework uses a novel optimization scheme and a two-stage strategy to find counterfactual videos near the original input, guided solely by the target classifier for faithfulness. Experiments on datasets for motion, emotion, and action classification demonstrate BTTF's ability to produce realistic counterfactual videos that offer insights into classifier decision-making. AI
IMPACT Provides a new method for understanding and debugging video classification models, potentially improving their reliability and trustworthiness.