Researchers have developed a new framework called Cognitive Diagnostic Profiling (CDP) to improve the psychometric calibration of educational tests using large language models (LLMs). CDP addresses the issue of LLM-simulated examinees being too accurate and uniform by prompting LLMs to simulate plausible examinees with diverse cognitive profiles. This approach was evaluated using the Tatsuoka dataset and showed significant improvements in aligning LLM responses with human examinees at the ability-distribution, mastery-profile, and item-difficulty levels. The framework brings LLM-simulated examinees closer to human behavior, making them a practical tool for test development. AI
IMPACT Enables more cost-effective and scalable development of educational assessments by leveraging LLMs for examinee simulation.
RANK_REASON The cluster contains an academic paper detailing a new methodology for using LLMs in psychometric calibration. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
