LLMs evaluated for citation function classification, achieving new SOTA

A new research paper evaluates several large language models (LLMs) for the task of citation function classification, aiming to improve bibliometric analysis. The study achieved new state-of-the-art results on the ACL-ARC dataset using a fine-tuned Falcon 7B model, reaching a 73.3% macro F1 score. Researchers also introduced AC3, a novel dataset with a seven-category annotation scheme to differentiate citation types, and explored the impact of context extraction variants on classification performance. AI

IMPACT This research could improve how scientific literature is analyzed and understood, potentially aiding researchers in discovering relevant connections and assessing the impact of papers.

RANK_REASON Academic paper presenting new dataset and benchmark results for LLM citation function classification. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• ACL-ARC
• Falcon 7B
• Hugging Face
• LLaMA 3.1-8B
• Mistral 7B
• Orca 2-7B
• SciBERT

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →