LLM annotation rivals human labels for hostility detection at lower cost

A new arXiv paper investigates the efficacy of Large Language Models (LLMs) in annotating data for active learning, specifically for hostility detection in online comments. The study found that LLMs, particularly GPT-5.2 with a two-question interface, can label data at a significantly lower cost than human annotators, achieving comparable or superior performance. However, the research also noted that active learning did not provide a reliable advantage over random sampling when using LLM annotators, and the error structures of different LLMs varied, with some misclassifying economic or border-control discourse. AI

IMPACT LLM annotation offers a cost-effective alternative to human labeling for specific tasks, potentially accelerating data annotation for AI development.

RANK_REASON The cluster contains an academic paper detailing research findings on LLM annotation. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• active learning
• Ahmad Dawar Hakimi
• arXiv
• German
• GPT-5.2
• Hostility Detection in Online Hindi-English Code-Mixed Conversations
• Hugging Face
• Prolific
• Qwen3.5-122B-10B
• TikTok

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →