PulseAugur
EN
LIVE 10:44:39

LLMs achieve 0.76 reliability in stance detection for Bayesian cognitive science

Researchers have developed a novel method for stance detection in scientific discourse, utilizing Large Language Models (LLMs) to analyze whether authors treat Bayesian models as descriptive mechanisms or useful mathematical tools. This approach combines a theory-driven codebook, expert annotations, and prompt optimization to achieve reliable zero-shot performance across frontier LLMs like GPT-5.1, Claude Sonnet 4.6, and Gemini 3 Pro Preview. The framework successfully quantified a long-held qualitative intuition that lower-level perception/motor articles exhibit higher realism scores than high-level cognition articles. AI

IMPACT This research provides a framework for using LLMs in nuanced qualitative analysis, potentially improving the scalability of scientific discourse analysis.

RANK_REASON The cluster contains an academic paper detailing a new methodology for LLM-assisted stance detection. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →

COVERAGE [1]

  1. arXiv cs.AI TIER_1 English(EN) · Eyup Engin Kucuk, Tarik Kelestemur, \"Omer Da\u{g}lar Tanrikulu ·

    LLM-Assisted Stance Detection in Scientific Discourse: A Test Case in Bayesian Cognitive Science

    arXiv:2606.15566v1 Announce Type: cross Abstract: Qualitative coding is central to social science, but expert annotation is difficult to scale. LLMs offer a possible extension, yet require careful validation when the target construct is interpretive, theoretically loaded, and onl…