Generalized Kullback-Leibler Divergence Loss
Researchers have introduced the Generalized Kullback-Leibler (GKL) Divergence loss, an enhancement to existing KL Divergence loss methods. This new loss function addresses limitations in scenarios like knowledge distillation by improving optimization for classes with high predicted scores and reducing sample bias. Experiments on datasets such as CIFAR-10/100, ImageNet, and vision-language tasks demonstrated GKL's effectiveness, achieving state-of-the-art adversarial robustness on RobustBench and competitive performance in knowledge distillation. AI
IMPACT Introduces a novel loss function that enhances adversarial robustness and knowledge distillation performance in AI models.