Power outage prediction models inflated by data leakage, study finds

A new research paper highlights significant data leakage issues in power outage prediction models, inflating their apparent generalizability. The study found that common methodological choices, particularly spatial and temporal autocorrelation, lead to overly optimistic performance metrics when evaluated using standard random train-test splits. When tested under more realistic conditions, such as leave-one-state-out or leave-one-event-out scenarios, the models' predictive accuracy degrades substantially, often failing to outperform simple baselines. The integration of GeoAI foundation model embeddings, like Prithvi WxC, offered only minor and inconsistent improvements, particularly for spatial generalization, and did not address poor event-level transferability. The findings suggest that current publicly trained models have limited operational value due to data limitations and evaluation practices, necessitating improved data coverage and more rigorous testing protocols. AI

IMPACT Highlights critical need for robust evaluation protocols in AI models to ensure real-world applicability.

RANK_REASON Research paper detailing methodological flaws in AI models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• alphaXiv
• CatalyzeX
• CORE Recommender
• DagsHub
• East Coast of the United States
• GeoAI
• Gotit.pub
• Hugging Face
• Influence Flower
• Prithvi WxC
• ScienceCast

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →