Researchers have introduced TSQueryBench, a new benchmark designed to evaluate how well large language models can assess the factual correctness of time series data explanations. The benchmark consists of 500 synthetic time series instances with associated explanations that are correct, partially correct, or incorrect. Experiments with six large language models revealed that while models struggle to generate numerically accurate explanations, they are reliably capable of identifying or scoring correct explanations when presented with them. This suggests that LLM-as-a-Judge evaluation methods are a viable and scalable approach for assessing numerically grounded time series reasoning. AI
IMPACT This benchmark could improve the reliability of LLM evaluations for time series data, crucial for high-stakes applications.
RANK_REASON The item describes a new academic paper introducing a novel benchmark for evaluating LLMs on a specific task. [lever_c_demoted from research: ic=1 ai=1.0]
- alphaXiv
- arXiv
- CatalyzeX
- DagsHub
- Gotit.pub
- Hugging Face
- LLM-as-a-Judge
- Preetham Sivalingam
- ScienceCast
- TSQueryBench
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
