VISReg: Variance-Invariance-Sketching Regularization for JEPA training
Researchers have introduced VISReg, a novel regularization technique for self-supervised learning in computer vision. This method enhances training stability by combining variance control with a Sliced-Wasserstein-based sketching objective, which enforces the full distributional shape of embeddings. VISReg demonstrates robust performance, outperforming existing methods on low-quality and out-of-distribution datasets, and achieves competitive results with significantly less data compared to other state-of-the-art approaches. AI
IMPACT Introduces a more robust regularization method for self-supervised learning, potentially improving performance on challenging datasets.