Catching The Correct Answer Trap: Characterising AI Tutor Blind Spots When Analysing Student Reasoning
Researchers have identified a significant failure mode in AI tutors, termed the "correct answer trap" (CAT), where systems fail to detect flawed student reasoning if the student arrives at the correct final answer. Analysis of student responses on the Eedi mathematics platform revealed that 71% of these CAT failures occurred in specific question types where incorrect reasoning coincidentally yielded the right numerical result. While advanced large language models showed improvement over fine-tuned T5 models in detecting these errors, they still struggled, with the best model only accurately identifying the flawed reasoning in 57% of cases and producing numerous false alarms, indicating that human oversight remains crucial for accurate assessment of student reasoning. AI
IMPACT AI tutors may require further development to accurately assess student reasoning, as current models can be misled by correct answers derived from flawed logic.