Researchers have introduced LEAF, a novel living benchmark designed to rigorously evaluate the forecasting capabilities of large language models (LLMs). LEAF addresses issues like data contamination and future information leakage by employing a recursive retrieval agent system and dual-agent cross-validation. Audits involving domain specialists revealed that LEAF significantly reduces future information leakage, and evaluations of 16 frontier LLMs demonstrated their effectiveness in using verified events to improve trend and event forecasting. AI
IMPACT Establishes a new standard for evaluating LLM forecasting capabilities, potentially driving improvements in model accuracy and reliability for time-series and event prediction.
RANK_REASON The cluster describes a new academic paper introducing a novel benchmark for evaluating LLMs. [lever_c_demoted from research: ic=1 ai=1.0]
- alphaXiv
- arXiv
- CatalyzeX
- CORE Recommender
- DagsHub
- Gotit.pub
- Hugging Face
- IArxiv Recommender
- Influence Flower
- large-language models
- LEAF
- Mingtian Tan
- ScienceCast
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
