LLM pipeline automates corpus annotation with 98% accuracy

Researchers have developed a novel four-phase pipeline for automating grammatical annotation in large natural language corpora using large language models (LLMs). This method, which includes prompt engineering, pre-hoc evaluation, batch processing, and post-hoc validation, achieved over 98% accuracy in annotating 143,933 'consider' concordance lines from the Corpus of Historical American English via the OpenAI API. A subsequent analysis revealed previously undocumented genre-specific changes in the evaluative consider construction, suggesting LLMs can significantly accelerate corpus linguistic research by enabling the exploration of questions previously out of practical reach. AI

IMPACT Enables large-scale linguistic research previously impractical due to manual annotation bottlenecks.

RANK_REASON The cluster describes a research paper detailing a new methodology for LLM-assisted corpus annotation. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• arXiv
• Cameron Morin
• Corpus of Contemporary American English
• Corpus of Historical American English
• Hugging Face
• OpenAI API

• paper
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →