New MMEarth-Bench dataset highlights poor geographic generalization in multimodal models

Researchers have introduced MMEarth-Bench, a new benchmark dataset designed to evaluate multimodal pretrained models in geospatial machine learning. The dataset features five tasks across 12 modalities with globally distributed data and includes both in- and out-of-distribution test splits. While existing multimodal pretraining shows promise for robustness in limited data scenarios, the benchmark reveals persistent weaknesses in geographic generalization. To address this, the study proposes a model-agnostic test-time training method (TTT-MMR) that leverages all available modalities for adaptation, showing improvements on both random and geographic test splits. AI

IMPACT Highlights limitations in current multimodal models for geographic generalization, potentially driving research into more robust adaptation techniques.

RANK_REASON The cluster contains an academic paper introducing a new benchmark dataset and a novel method for multimodal test-time training. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• MMEarth-Bench

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →