Self-Soupervision: Cooking Model Soups without Labels
Researchers have developed a new method called Self-Soupervision, which allows for the creation of "model soups" using self-supervised learning (SSL) instead of traditional supervised learning. This technique enables the combination of parameters from multiple models, even those trained with different SSL algorithms or hyperparameters, to enhance prediction accuracy and robustness. Experiments demonstrated that Self-Souping improved robustness on corrupted datasets like ImageNet-C and LAION-C, and successfully created soups of diverse SSL ingredients that outperformed individual models. AI
IMPACT Enables more robust and accurate models by leveraging unlabeled data, potentially reducing reliance on expensive labeled datasets.