Researchers have developed a new protocol for generating and evaluating codebooks using large language models (LLMs) to measure the quality of parliamentary discourse. This method aims to overcome the limitations of traditional expert-scored codebooks, which are time-consuming and potentially biased. The proposed approach compares top-down, bottom-up, and hybrid methods for creating context-adapted schemas, introducing an extrinsic proxy measure and an intrinsic inter-model agreement (IMA) metric for evaluation. Experiments on Israeli parliament transcripts showed that data-driven schemas achieved higher IMA than theory-based methods while retaining validity, suggesting their utility for large-scale political discourse analysis. AI
IMPACT This research could enable more scalable and less biased analysis of political discourse, potentially informing policy and public understanding.
RANK_REASON The cluster contains a research paper detailing a new methodology for evaluating discourse quality using LLMs. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
