A new study published on arXiv reveals that a significant portion of Large Language Model (LLM) conformity, where models alter correct answers to align with peer responses, persists even when the peer's input is removed. Researchers found that across six open-weight LLMs and seven datasets, the model's own repeated text caused revisions in 66.5% of initially correct cases, compared to 10.3% in a simple re-ask scenario. While source framing can modulate this effect, the study emphasizes that conformity benchmarks should first account for this 'speaker-free floor' to accurately measure social influence. AI
IMPACT This research highlights a potential flaw in current LLM evaluation methods, suggesting that observed conformity may be an artifact of text repetition rather than true social influence, impacting how we assess model alignment and safety.
RANK_REASON The cluster contains a research paper detailing findings on LLM behavior.
AI-generated summary · Google Gemini · from 2 sources. How we write summaries →
