Researchers have developed a novel method for automatically testing the accuracy of explanations generated by large language models (LLMs) for sequential decision-making policies. This approach utilizes probabilistic model checking as an oracle to evaluate the faithfulness of LLM-generated explanations against the underlying environment. By structuring test inputs based on a taxonomy of post hoc query categories and prioritizing test cases by diagnostic difficulty, the system can systematically assess LLM performance. Experiments across seven environments showed that a reasoning model achieved 85% accuracy, a mid-size model 70%, and a 1B model performed below the random baseline, highlighting the varying trustworthiness of LLM explanations in model-free settings. AI
IMPACT This research provides a framework for evaluating the reliability of LLM-generated explanations, crucial for applications requiring trustworthy decision-making insights.
RANK_REASON The cluster contains an academic paper detailing a new methodology for testing LLM explainers. [lever_c_demoted from research: ic=1 ai=1.0]
- 1B model
- arXiv
- Hugging Face
- LLM
- Markov decision processes
- mid-size model
- model checking
- reasoning language model
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
