New LocaLeJEPA framework improves self-supervised learning with geometric priors

Researchers have introduced LocaLeJEPA, a new framework for self-supervised learning that addresses representation collapse issues in Joint-Embedding Predictive Architectures (JEPA). Unlike previous methods tied to Euclidean geometry, LocaLeJEPA incorporates domain-specific geometric priors by reformulating JEPA's entropy-maximization property using manifold theory. The accompanying LocaLeReg regularizer enables a stable implementation, and experiments on ImageNet-1K show LocaLeJEPA outperforms its predecessor, LeJEPA, across various backbone models and downstream tasks. AI

IMPACT Enhances self-supervised learning by better respecting intrinsic data geometry and accommodating exotic embedding geometries.

RANK_REASON The cluster contains a research paper detailing a new framework for self-supervised learning. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• arXiv
• hyperbolic ViT-S/8
• ImageNet
• ImageNet-1K
• LeJEPA
• LocaLeJEPA
• LocaLeReg
• ResNet-50
• ViT-L-14
• ViT-S/8

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →