Multi-Modal Contrastive Learning for Implicit Earth Embeddings via Location Tying
Researchers have developed two novel multimodal contrastive learning architectures, MELT and SALT, designed to improve the encoding of location-based data. These methods aim to overcome limitations in spatial prediction tasks by utilizing unpaired geospatial data across multiple modalities, moving beyond the typical two-modality approach. While both MELT and SALT achieve performance comparable to existing two-modality baselines like SATCLIP on four downstream tasks, the study suggests that the location encoder itself is the primary bottleneck for further improvement, rather than the diversity or volume of modalities. MELT demonstrates more consistent training stability compared to SALT, positioning it as a more promising avenue for future scaling efforts. AI
IMPACT These methods could enhance spatial prediction tasks by improving the encoding of location-based data through multimodal learning.