New Generalized KL Divergence Loss Achieves State-of-the-Art Robustness

Researchers have introduced the Generalized Kullback-Leibler (GKL) Divergence loss, an enhancement to existing KL Divergence loss methods. This new loss function addresses limitations in scenarios like knowledge distillation by improving optimization for classes with high predicted scores and reducing sample bias. Experiments on datasets such as CIFAR-10/100, ImageNet, and vision-language tasks demonstrated GKL's effectiveness, achieving state-of-the-art adversarial robustness on RobustBench and competitive performance in knowledge distillation. AI

IMPACT Introduces a novel loss function that enhances adversarial robustness and knowledge distillation performance in AI models.

RANK_REASON The cluster contains an academic paper detailing a new loss function for machine learning models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• CIFAR-10/100
• Cui Jiequan
• Decoupled Kullback-Leibler Divergence
• Generalized Kullback-Leibler Divergence
• ImageNet
• Kullback-Leibler Divergence
• RobustBench

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →