Researchers have conducted a large-scale study on scaling pixel-wise Earth-observation foundation models, involving 395 training runs on 1,024 GH200 superchips. The study found that pretraining loss is a poor predictor of downstream performance, leading to wasted compute when models are selected solely on this metric. A key finding is that as training budgets increase, the encoder and data should scale together, while the projector remains fixed, offering a clear method for allocating computational resources. The team trained distilled models, with the 21-million-parameter TESSERA v2-1B-M outperforming larger open and proprietary models on aggregate, offering Matryoshka representations for efficient deployment. AI
IMPACT Provides an empirically grounded recipe for scaling pixel-wise Earth-observation foundation models, potentially improving efficiency and performance in geospatial AI applications.
RANK_REASON Academic paper detailing a scaling study and new methodology for foundation models. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
