New framework uses SHAP and LLMs to explain teaching quality scores

Researchers have developed a new framework to interpret how automated scoring models assign quality ratings to complex language performances, such as classroom transcripts. This framework combines model-agnostic Shapley-value attributions with explanations generated by large language models (LLMs). In tests on the CLASS framework's Quality of Feedback dimension, Shapley values proved more reliable and transferable than LLM-generated rationales for explaining model predictions. AI

IMPACT Provides a more robust method for evaluating the faithfulness and transferability of explanations from AI models in educational assessment.

RANK_REASON The cluster contains an academic paper detailing a new framework for evaluating LLM rationales and SHAP for rubric-based assessment. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• CLASS framework
• LLMs
• NCTE corpus
• SHAP

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →