PulseAugur
EN
LIVE 08:00:34

LLM responses to pseudo-science are unstable and opaque, study finds

A new paper reveals that major LLMs like Claude, Grok, Gemini, and GPT exhibit unstable and opaque responses to pseudo-scientific claims. Grok's Fast versions, powering X, consistently rated ethnonationalist pseudo-science much higher than other models. The study found that LLM responses are not stable properties of the model itself but are heavily influenced by deployment configurations such as system prompts and silent updates, making their epistemic stance unpredictable and a matter of public concern. AI

IMPACT Highlights the need for greater transparency and accountability in LLM deployment, impacting how users and researchers trust AI-generated information.

RANK_REASON The cluster contains an academic paper detailing research findings on LLM behavior. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →

LLM responses to pseudo-science are unstable and opaque, study finds

COVERAGE [1]

  1. arXiv cs.CL TIER_1 English(EN) · Davide Scarso, Hugo Noronha de Almeida, Joaquim Pina ·

    Opaque Epistemic Mediation: How LLM Deployment Configurations Shape the Validation of Pseudo-Science

    arXiv:2607.22513v1 Announce Type: cross Abstract: Commercial large language models are increasingly used as knowledge references, yet their stance on contested scientific claims is neither stable nor transparent. We tested how four major LLM families (Claude, Grok, GPT, Gemini) e…