Pretraining objective impacts low-data image classification

A new study on arXiv investigates the impact of different pretraining objectives on the performance of visual encoders in extreme low-data fine-grained classification tasks. Researchers compared four frozen ViT-B/16 encoders trained with supervised classification, contrastive learning (SigLIP2), masked reconstruction (MAE), and self-distillation (DINOv3) using a custom dataset of emerald inclusion images. The findings indicate that supervised and contrastive learning methods yield the strongest results for linear separability, while MAE performs better with nonlinear probes. DINOv3 was found to underperform in this specific domain. AI

Summary written by gemini-2.5-flash-lite from 1 source. How we write summaries →

IMPACT Provides practical guidance for selecting pretraining methods in data-scarce fine-grained visual classification scenarios.

RANK_REASON Academic paper detailing a controlled study on model pretraining objectives. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• arXiv
• DINOv3
• MAE
• SigLIP2
• ViT-B/16

• paper
• other