Crystal uses LLMs to rank scholarly paper impact

Researchers have developed Crystal, a new method that uses large language models to assess the impact of scholarly publications by analyzing their citation context. Unlike previous methods that evaluate citations in isolation, Crystal jointly ranks all cited papers within a single document, allowing for relative comparisons. This approach mitigates LLM positional bias through randomized ranking and aggregation, leading to improved accuracy and efficiency. Crystal has demonstrated superior performance compared to existing state-of-the-art classifiers and aligns with long-term scientific recognition, with a new dataset and code released for public use. AI

IMPACT Introduces a novel LLM-based methodology for evaluating academic research impact, potentially improving scholarly discovery and assessment.

RANK_REASON The cluster describes a new method presented in a scholarly paper for analyzing citation impact using LLMs. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• Crystal
• Hannah Collison
• Large Language Models

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →