LLM-Assisted Stance Detection in Scientific Discourse: A Test Case in Bayesian Cognitive Science
Researchers have developed a novel method for stance detection in scientific discourse, utilizing Large Language Models (LLMs) to analyze whether authors treat Bayesian models as descriptive mechanisms or useful mathematical tools. This approach combines a theory-driven codebook, expert annotations, and prompt optimization to achieve reliable zero-shot performance across frontier LLMs like GPT-5.1, Claude Sonnet 4.6, and Gemini 3 Pro Preview. The framework successfully quantified a long-held qualitative intuition that lower-level perception/motor articles exhibit higher realism scores than high-level cognition articles. AI
IMPACT This research provides a framework for using LLMs in nuanced qualitative analysis, potentially improving the scalability of scientific discourse analysis.