Researchers have developed a new protocol called Disengagement-Aware Student Simulators (DAS2) to evaluate AI tutors by modeling five learner-engagement states: engaged, gaming, wheel-spinning, off-task, and mixed. This protocol aims to assess how AI tutors respond to disengaged student behaviors, which is crucial for effective pedagogical strategies. Evaluations using the ASSISTments09 dataset showed that DAS2 significantly reduced the performance gap between simulated and authentic tutoring sessions for gaming and wheel-spinning states. While fine-tuned Qwen2.5-7B better matched authentic response times, prompt-only GPT-4o generated more distinguishable learner states. The study also found that relative rankings of AI tutors remained stable across different learner states, but absolute performance varied, highlighting the need for state-specific tutor support. AI
IMPACT This research provides a framework for more robust AI tutor evaluation, potentially leading to more effective and adaptive educational tools.
RANK_REASON Academic paper detailing a new evaluation protocol for AI tutors. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
