New method enhances deep neural network interpolation robustness

Researchers have introduced Sharp Mode Connectivity (SMC), a new method for optimizing parametric curves in the weight space of deep neural networks. Unlike standard mode connectivity, which only ensures low loss along a trajectory, SMC enforces low loss in the surrounding neighborhood as well, making the interpolated models more robust to distribution shifts. This approach, validated on models like ResNet-18 and ViT-Tiny across datasets such as CIFAR-10 and ImageNet-100, demonstrated significant accuracy improvements, particularly under corruptions like CIFAR-10-C, and even produced negative loss barriers. AI

IMPACT Enhances robustness of interpolated models, potentially improving techniques like weight averaging and model merging.

RANK_REASON Academic paper introducing a novel method for deep neural network interpolation. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• CIFAR-10
• CIFAR-10-C
• ImageNet-100
• mode connectivity
• ResNet-18
• SAM-on-the-Curve
• Sharpness-Aware Mode Connectivity
• VGG16 BN
• ViT-Tiny

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →