Power outage prediction models inflated by data leakage, study finds

A new research paper highlights significant data leakage issues in power outage prediction models, inflating their apparent generalizability. The study found that common methodological choices, particularly spatial and temporal autocorrelation, lead to overly optimistic performance metrics when evaluated using standard random train-test splits. When tested under more realistic conditions, such as leave-one-state-out or leave-one-event-out scenarios, the models' predictive accuracy degrades substantially, often failing to outperform simple baselines. The integration of GeoAI foundation model embeddings, like Prithvi WxC, offered only minor and inconsistent improvements, particularly for spatial generalization, and did not address poor event-level transferability. The findings suggest that current publicly trained models have limited operational value due to data limitations and evaluation practices, necessitating improved data coverage and more rigorous testing protocols. AI

影响 Highlights critical need for robust evaluation protocols in AI models to ensure real-world applicability.

排序理由 Research paper detailing methodological flaws in AI models. [lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.LG 阅读 →

• alphaXiv
• CatalyzeX
• CORE Recommender
• DagsHub
• East Coast of the United States
• GeoAI
• Gotit.pub
• Hugging Face
• Influence Flower
• Prithvi WxC
• ScienceCast

• 论文
• 安全

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →