SMI: Efficient Self-Supervised Learning via Mutual-Information-Inspired Dependency Optimization
Two new research papers explore novel approaches to self-supervised learning (SSL) in computer vision, aiming to improve efficiency and performance. The first paper introduces Semantic Mutual Information (SMI), a method that optimizes a sample-level dependency matrix to achieve competitive results with reduced computational cost. The second paper proposes a multi-task formulation for Siamese SSL, assigning a dedicated predictor to each spatial transformation to stabilize optimization and enhance performance across different frameworks. AI
IMPACT These papers introduce novel techniques that could lead to more efficient and effective computer vision models, potentially reducing training costs and improving performance on various downstream tasks.