Researchers have introduced BAIT (Bias Annotation In Text), a new resource designed to detect cognitive bias triggers within news articles. The resource includes a schema of nine bias triggers, an annotator codebook, and 50 annotated English news articles. When tested on six large language models, the LLMs showed performance comparable to humans in identifying the presence of triggers at the article level but struggled significantly in pinpointing the exact passages containing these biases. The models also tended to over-predict triggers based on superficial cues, indicating a reliance on surface-level features rather than a deep understanding of manipulative mechanisms. AI
IMPACT This benchmark highlights current LLM limitations in understanding nuanced linguistic manipulation, suggesting a need for improved bias detection capabilities in AI news summarization and analysis tools.
RANK_REASON The cluster describes a new academic paper introducing a benchmark for evaluating LLM capabilities in detecting cognitive bias in news articles. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
