A new benchmark called FinDendro has been developed to test the temporal self-knowledge of large language models (LLMs) in financial backtesting. The benchmark, comprising 2,226 dated U.S. facts from 1999 to 2026, reveals that LLMs often understate their knowledge when asked about past events. For instance, GPT-5 answered 76% of controlled facts correctly but claimed knowledge of only 38%, and even then, it answered 61% of the facts it claimed not to know. The study indicates that self-reporting, instructions to answer from a past date, or stated knowledge cutoffs are insufficient to guarantee leak-free financial backtests. AI
IMPACT Highlights the unreliability of LLM self-reporting for temporal knowledge, impacting the trustworthiness of AI in time-sensitive financial applications.
RANK_REASON The cluster contains a research paper introducing a new benchmark for evaluating LLM capabilities. [lever_c_demoted from research: ic=1 ai=1.0]
- alphaXiv
- arXiv
- CatalyzeX
- DagsHub
- Federal Reserve System
- FinDendro
- Gotit.pub
- GPT-5
- Hugging Face
- ScienceCast
- U.S.
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
