LLM "memory shortcuts" can invalidate annotations when text evidence is missing

A new research paper explores the phenomenon of "memory shortcuts" in large language models (LLMs) when annotating text. The study found that while LLMs generally do not substitute their existing knowledge for textual evidence when the two conflict, they do exhibit these shortcuts when evidence is absent. This can lead to annotations that are factually correct about the subject but not supported by the provided text, potentially invalidating research. The paper suggests that trustworthy LLM annotation requires audits to test for evidence absence, consider annotation instruction incentives, and evaluate mitigation trade-offs. AI

IMPACT Highlights potential methodological flaws in LLM-assisted research and necessitates new auditing techniques for reliable annotation.

RANK_REASON The cluster contains a single academic paper detailing research findings on LLM behavior. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• alphaXiv
• arXiv
• CatalyzeX Code Finder for Papers
• Connected Papers
• DagsHub
• Gotit.pub
• Hugging Face
• human–computer interaction
• large language models
• Litmaps
• ScienceCast
• scite Smart Citations
• When the Text Is Silent: Memory Shortcuts and the Validity of LLM Annotation under Evidence Insufficiency

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →