LLM framework enhances similarity analysis for large-scale assessments

Researchers have developed a new framework called AISA, which utilizes Large Language Models (LLMs) to analyze incidental content similarity in large-scale assessments. This dual-dimensional approach, operationalizing similarity through structured decomposition and semantic relatedness, aims to address the limitations of traditional metrics like BLEU and cosine similarity in capturing nuanced redundancy. Psychometric validation suggests that AISA's LLM-derived metrics better align with construct-irrelevant local dependence and improve item parameter groupings. The framework's application in Computerized Adaptive Testing (CAT) simulations demonstrated enhanced estimation stability and reduced bias in item selection compared to conventional methods. AI

影响 This LLM-powered framework could improve the quality and efficiency of creating large-scale assessments and adaptive tests.

排序理由 The cluster contains an academic paper detailing a new framework and its validation. [lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.AI 阅读 →

• Aisa
• arXiv
• BLEU
• Computerized Adaptive Testing
• cosine similarity
• Large Language Models
• LLMs

• 论文
• 产品

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →